* [Fuego] New batch of patches
@ 2017-05-12 8:19 Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency Daniel Sangorrin
` (19 more replies)
0 siblings, 20 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Hi Tim,
Please consider this patches for fuego-core. Some of them are
a refinement over the previous patches that I sent in a rush, and
some of them are new. I have kept the concurrent_check function
for now.
[PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for
[PATCH 02/19] functions:fail_check_cases: add message when no fail
This one originally included a fix like this:
# expr returns 1 if the expression is 0, which may be incompatible with set -e
- fcc=`expr $fcc - 1` || true
But I removed it because it caused a merge conflict with your new
patch. I think the problem doesn't arise with your patch.
[PATCH 03/19] ft2demos: fix test and add to docker
[PATCH 04/19] unpack: remove nostrip option
[PATCH 05/19] unpack: move code related to spec-defined tarballs to
[PATCH 06/19] bc: test that specs can override the tarball variable
[PATCH 07/19] unpack: handle possible errors
[PATCH 08/19] main: remove ReBuild check
[PATCH 09/19] rebuild: make it simpler for test developers
This is the biggest change in this series, but I think it solves one
of those problems that you mentioned when you see the same pattern
repeated all over that should be shared somehow. You were right
on your concerns about the LTP test not being able to handle the
flag, but I fixed that PATCH 15/19.
[PATCH 10/19] call_if_present: tests do not need to implement dummy
You mentioned that you liked to have all functions as in a
template, but I think this is also one of those patterns and
we should remove them. Instead we should provide a nice template
in the documentation.
[PATCH 11/19] expat: remove unneeded semicolons
This test is not working on docker for some reason. The OpenSSH
test doesn't work either on docker (but I think it does on my ARM board).
We might need to double-check or remove them from testplan_docker.
[PATCH 12/19] tarball: remove unneeded tarball definitions
[PATCH 13/19] style: fix trailing spaces and indentation
[PATCH 14/19] parser: allow parser.py to skip processing
This is not very beautiful but it's required (see PATCH 16/19).
[PATCH 15/19] LTP: copy target_bin before modifying it
[PATCH 16/19] LTP: fix the buildonly and runonly cases
[PATCH 17/19] LTP: add support for skipping certain test cases
This implements the skiplist (previously known as blacklist but
I call it skiplist like inside the LTP project).
[PATCH 18/19] LTP: add a spec for docker
[PATCH 19/19] LTP: add a fixthis for the -t flag
This could be useful for handling runaway processes that hang
the CPU for ours.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 22:56 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 02/19] functions:fail_check_cases: add message when no fail cases available Daniel Sangorrin
` (18 subsequent siblings)
19 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
I also improved the documentation of the function fail_check_cases
to mention that syslog (actually the diff) may also be checked
by defining the variable use_syslog on the spec.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 7 +++++--
engine/scripts/ovgen.py | 2 +-
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index 1432b5e..f9681b3 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -393,7 +393,10 @@ function processing {
return $RETURN_VALUE
}
-# search in test log for {!JOB_NAME}_FAIL_PATTERN_n fail cases and abort with message {!JOB_NAME}_FAIL_MESSAGE_n if found
+# search in testlog.txt and syslog (if use_syslog is defined in the spec)
+# for {!TESTDIR}_FAIL_REGEXP_n fail cases (fail_regexp in the spec) and
+# abort with message {!TESTDIR}_FAIL_MESSAGE_n (fail_message in the spec)
+# if found.
function fail_check_cases () {
testlog="${LOGDIR}/testlog.txt"
slog_prefix="${LOGDIR}/syslog"
@@ -413,7 +416,7 @@ function fail_check_cases () {
for n in `seq 0 $fcc`
do
- fpvarname="${upName}"_FAIL_PATTERN_"${n}"
+ fpvarname="${upName}"_FAIL_REGEXP_"${n}"
fpvarmsg="${upName}"_FAIL_MESSAGE_"${n}"
fpvarslog="${upName}"_FAIL_"${n}"_SYSLOG
diff --git a/engine/scripts/ovgen.py b/engine/scripts/ovgen.py
index e514d7e..ed67667 100755
--- a/engine/scripts/ovgen.py
+++ b/engine/scripts/ovgen.py
@@ -422,7 +422,7 @@ def generateSpec(ts, fout):
fout.write(outNum + "\n")
for fmsg, num in zip(ts.fail_case, range(fc_num)):
- outPattern = "%s_FAIL_PATTERN_%s=\"%s\"" % (tNameUp, num, fmsg["fail_regexp"])
+ outPattern = "%s_FAIL_REGEXP_%s=\"%s\"" % (tNameUp, num, fmsg["fail_regexp"])
outMessage = "%s_FAIL_MESSAGE_%s=\"%s\"" % (tNameUp, num, fmsg["fail_message"])
if "use_syslog" in fmsg:
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 02/19] functions:fail_check_cases: add message when no fail cases available
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 03/19] ft2demos: fix test and add to docker Daniel Sangorrin
` (17 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
TODO (actually fixed by Tim in his branch):
expr returns 1 if the expression is 0 (e.g. when we have a single
fail_regexp/fail_message tuple), which may be incompatible with set -e.
Return True in case it fails.
Note, expr can also return 2 if the expression is syntactically
invalid and 3 if an error occurred. These return values are
ignored here because they are not expected to happen in Fuego.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 2 ++
1 file changed, 2 insertions(+)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index f9681b3..ac19e8e 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -442,6 +442,8 @@ function fail_check_cases () {
return 1
fi
done
+
+ echo "No fail cases detected for $JOB_NAME"
}
# $@ are process names to kill on the target
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 03/19] ft2demos: fix test and add to docker
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 02/19] functions:fail_check_cases: add message when no fail cases available Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 04/19] unpack: remove nostrip option Daniel Sangorrin
` (16 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
This was the only test that required the flag nostrip for
unpacking. That was too obscure, so I just unpack the 2
tarballs during test_build. For clarity/readability I also
avoided hard to understand shell script string extractions.
I have tested it on docker and it works. I tried on an ARM
board but I got some errors during the build phase. I believe
these errors are because the tarballs themselves are not
cross-compile friendly and because my toolchain lacked some
of the required libraries.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/overlays/testplans/testplan_docker.json | 3 ++
engine/tests/Functional.ft2demos/fuego_test.sh | 43 ++++++++++----------------
2 files changed, 19 insertions(+), 27 deletions(-)
diff --git a/engine/overlays/testplans/testplan_docker.json b/engine/overlays/testplans/testplan_docker.json
index 6336d36..e2e5d14 100644
--- a/engine/overlays/testplans/testplan_docker.json
+++ b/engine/overlays/testplans/testplan_docker.json
@@ -78,6 +78,9 @@
"testName": "Functional.ipv6connect"
},
{
+ "testName": "Functional.ft2demos"
+ },
+ {
"testName": "Functional.linus_stress",
"timeout": "20m"
},
diff --git a/engine/tests/Functional.ft2demos/fuego_test.sh b/engine/tests/Functional.ft2demos/fuego_test.sh
index 1f04447..99e672e 100755
--- a/engine/tests/Functional.ft2demos/fuego_test.sh
+++ b/engine/tests/Functional.ft2demos/fuego_test.sh
@@ -1,39 +1,28 @@
-tarball=ft2demos-2.3.6.tar.bz2
-
function test_build {
- # ft2demos depends on already built freetype2. Moreover,
- # it is expected that freetype2 directory is located on the same
- # level with ft2demos. Hence, nostrip flag.
- # Freetype2
-
- # A2 variable represents FT2 version
- a2=${tarball/*-}
- # Original Freetype2 package w/o tests.
- ft_tarball=freetype-${a2%.*.*}.tar.bz2
-
- tar jxf "$TEST_HOME/$ft_tarball"
- mv ${ft_tarball%.*.*} freetype2
-
- cd freetype2
- ./configure --build=`uname -m`-linux-gnu --host=$HOST $OPTIONS CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" --target="$HOST"
- make
-
- # Freetype2 Demos
- cd ../${tarball%.tar*}
- patch -N -s -p0 < $TEST_HOME/ft2demos.Makefile.patch
- export PATH=/usr/local/bin/:$PATH
- SDKROOT=$SDKROOT CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP LD=$LD make && touch ../test_suite_ready || build_error "error while building test"
+ # ft2demos (Freetype2 Demos) depends on freetype
+ tar jxvf $TEST_HOME/freetype-2.3.6.tar.bz2
+ cd freetype-2.3.6/
+ ./configure --build=`uname -m`-linux-gnu --host=$HOST $OPTIONS CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" --target="$HOST" || build_error "error while configuring freetype"
+ make || build_error "error while building freetype"
+
+ # build ft2demos
+ cd ..
+ tar jxvf $TEST_HOME/ft2demos-2.3.6.tar.bz2
+ cd ft2demos-2.3.6/
+ patch -N -s -p0 < $TEST_HOME/ft2demos.Makefile.patch || build_error "error while patching ft2demos"
+ make TOP_DIR=../freetype-2.3.6 || build_error "error while building ft2demos"
}
function test_deploy {
- put ${tarball%.tar*}/bin/.libs/* $BOARD_TESTDIR/fuego.$TESTDIR/
+ put ft2demos-2.3.6/bin/.libs/* $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ls -1 /usr/share/fonts/truetype/* | xargs -n1 ./ftdump $1"
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; find /usr/share/fonts/truetype/ -iname '*.ttf' | xargs -n1 ./ftdump $1"
}
function test_processing {
- log_compare "$TESTDIR" "12" ".*family:" "p"
+ local count=$(find /usr/share/fonts/truetype/ -iname '*.ttf' | wc -l)
+ log_compare "$TESTDIR" "$count" ".*family:" "p"
log_compare "$TESTDIR" "0" "fail|error|FAIL|ERROR" "n"
}
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 04/19] unpack: remove nostrip option
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (2 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 03/19] ft2demos: fix test and add to docker Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 22:57 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 05/19] unpack: move code related to spec-defined tarballs to unpack Daniel Sangorrin
` (15 subsequent siblings)
19 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
unpack used to have a "nostrip" option so that the tarball
would be extracted to its own folder. This was used in the past
by ft2demos which was calling build directly.
The problem is that tests are not supposed to call build directly
and therefore we would need to add yet another environment variable
for main.sh to pass it to build. Since most tests are fine with the
current default options, I think that tests like ft2demos should
unpack their own tarballs during test_build instead.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index ac19e8e..d9e0b18 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -37,11 +37,7 @@ trap signal_handler SIGTERM SIGHUP SIGALRM SIGINT EXIT
set -o errtrace
# Unpacks $tarball_path/$tarball into current directory.
-# $1 - optional flag; if set to "nostrip",
-# the leading path components won't be stripped
function unpack {
- [ "$1" = "nostrip" ] && strip_opt= || strip_opt="--strip-components=1"
-
if [ ! -z ${tarball+x} ]; then
case ${tarball/*./} in
gz|tgz) key=z ;;
@@ -49,14 +45,15 @@ function unpack {
tar) key= ;;
*) echo "Unknown $tarball file format. Not unpacking."; return;;
esac
- tar ${key}xf $TEST_HOME/$tarball $strip_opt
+ tar ${key}xf $TEST_HOME/$tarball --strip-components=1
fi
if [ ! -z ${gitrepo+x} ]; then
if [ -z ${gitbranch+x} ]; then
gitbranch="master"
fi
- # FIXTHIS: support commit ids (log and checkout)
+ # FIXTHIS: support commit ids (log and checkout)
+ # FIXTHIS: strip components like tarballs
git clone --depth=1 --branch=$gitbranch $gitrepo
fi
}
@@ -171,7 +168,6 @@ function build_error () {
# process Rebuild flag, and unpack test sources if necessary.
# Returns 0 if actual build needs to be performed; 1 - otherwise.
# Build scripts must call this function in the beginning.
-# $1 is passed directly to unpack().
function pre_build {
cd ${WORKSPACE}
upName=`echo "${JOB_NAME^^}"| tr '.' '_'`
@@ -193,7 +189,7 @@ function pre_build {
fi
if [ ! -e test_suite_ready ]; then
- unpack $1
+ unpack
return 0
fi
fi
@@ -201,7 +197,7 @@ function pre_build {
}
function build {
- pre_build $1
+ pre_build
build_start_time=$(date +"%s.%N")
test_build || return 1
build_end_time=$(date +"%s.%N")
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 05/19] unpack: move code related to spec-defined tarballs to unpack
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (3 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 04/19] unpack: remove nostrip option Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 23:05 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 06/19] bc: test that specs can override the tarball variable Daniel Sangorrin
` (14 subsequent siblings)
19 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Test specs can override the default tarball by specifying
a new one with tarball_name. This code should be inside unpack
for clarity.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index d9e0b18..9eae52b 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -38,6 +38,16 @@ set -o errtrace
# Unpacks $tarball_path/$tarball into current directory.
function unpack {
+ # the tarball name can be defined in the spec (tarball_name) or direclty
+ # in fuego_test.sh (tarball). Note that the one in the spec has preference.
+ upName=`echo "${TESTDIR^^}"| tr '.' '_'`
+ spec_tarball="${upName}_TARBALL_NAME"
+
+ if [ ! -z "${!spec_tarball}" ]; then
+ echo "Using $spec_tarball=${!spec_tarball} from test spec"
+ tarball=${!spec_tarball}
+ fi
+
if [ ! -z ${tarball+x} ]; then
case ${tarball/*./} in
gz|tgz) key=z ;;
@@ -170,14 +180,6 @@ function build_error () {
# Build scripts must call this function in the beginning.
function pre_build {
cd ${WORKSPACE}
- upName=`echo "${JOB_NAME^^}"| tr '.' '_'`
- spec_tarball="${upName}_TARBALL_NAME"
-
- if [ ! -z "${!spec_tarball}" ]
- then
- echo "Using $spec_tarball=${!spec_tarball} from test spec"
- tarball=${!spec_tarball}
- fi
mkdir -p $TRIPLET && cd $TRIPLET
if concurrent_check; then
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 06/19] bc: test that specs can override the tarball variable
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (4 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 05/19] unpack: move code related to spec-defined tarballs to unpack Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 07/19] unpack: handle possible errors Daniel Sangorrin
` (13 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
This commit adds a new tarball (by2.tar.gz) for Functional.bc
that just multiplies by 2 whatever expression is provided.
It is introduced to demonstrate that the tarball variable
definition can be overriden from the spec by defining
an alternative tarball through the 'tarball_name' key.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/overlays/testplans/testplan_docker.json | 4 ++++
engine/tests/Functional.bc/by2.tar.gz | Bin 0 -> 264 bytes
engine/tests/Functional.bc/spec.json | 5 +++++
3 files changed, 9 insertions(+)
create mode 100644 engine/tests/Functional.bc/by2.tar.gz
diff --git a/engine/overlays/testplans/testplan_docker.json b/engine/overlays/testplans/testplan_docker.json
index e2e5d14..488417d 100644
--- a/engine/overlays/testplans/testplan_docker.json
+++ b/engine/overlays/testplans/testplan_docker.json
@@ -95,6 +95,10 @@
"testName": "Functional.bc"
},
{
+ "testName": "Functional.bc",
+ "spec": "bc-by2"
+ },
+ {
"testName": "Functional.zlib"
},
{
diff --git a/engine/tests/Functional.bc/by2.tar.gz b/engine/tests/Functional.bc/by2.tar.gz
new file mode 100644
index 0000000000000000000000000000000000000000..a424d5b9cfe03497709a12d21d65886f43ba9730
GIT binary patch
literal 264
zcmb2|=3w~0TPTu&`R&DnUd)C9Y!Bu<*F3g^U9@C!&SuNEj!T;&0#-EzbmeZ-+#R4b
zLyN<1&+d$+6&sejcKt7(X)sgK`PlxMAtgShbGGJQIk)QU4z-eHq36?%J&T-N?k9P0
z+v|f-`h}*oX=0mq{Medz`)}W;v;S%<ZMfH$O8O;TJt}?sTYkvCd;3c({Wb2U6dhdE
zomltl`S1Jpy=(XS{51)4Nq^q|w|#cv#24Qz-rT>(|E}wZz|~^8-U^{puR5PPehMwG
zNKx6pSzkRyL}&7Qxs#6#*RI;2l2=m}8}G^0EnyZUzI-~f=jT~3*4HyKpn(MQT81-L
KF)J7}7#IM@hkH)|
literal 0
HcmV?d00001
diff --git a/engine/tests/Functional.bc/spec.json b/engine/tests/Functional.bc/spec.json
index f62eae5..4ce6ace 100644
--- a/engine/tests/Functional.bc/spec.json
+++ b/engine/tests/Functional.bc/spec.json
@@ -23,6 +23,11 @@
"default": {
"EXPR":"3+3",
"RESULT":"6"
+ },
+ "bc-by2": {
+ "tarball_name": "by2.tar.gz",
+ "EXPR":"3+3",
+ "RESULT":"12"
}
}
}
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 07/19] unpack: handle possible errors
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (5 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 06/19] bc: test that specs can override the tarball variable Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 23:09 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 08/19] main: remove ReBuild check Daniel Sangorrin
` (12 subsequent siblings)
19 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Tests can use source code from a tarball or from a git repository.
Differentiate both cases with return sentences and handle
possible errors.
Note that there are tests that do not require any source code
so make sure they do not cause an error as well.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index 9eae52b..d5bf969 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -36,7 +36,7 @@ trap signal_handler SIGTERM SIGHUP SIGALRM SIGINT EXIT
# cause ERR trap setting to be visible inside functions
set -o errtrace
-# Unpacks $tarball_path/$tarball into current directory.
+# Unpacks/clones the test source code into the current directory.
function unpack {
# the tarball name can be defined in the spec (tarball_name) or direclty
# in fuego_test.sh (tarball). Note that the one in the spec has preference.
@@ -53,9 +53,10 @@ function unpack {
gz|tgz) key=z ;;
bz2) key=j ;;
tar) key= ;;
- *) echo "Unknown $tarball file format. Not unpacking."; return;;
+ *) echo "Unknown $tarball file format. Not unpacking."; return 1;;
esac
tar ${key}xf $TEST_HOME/$tarball --strip-components=1
+ return
fi
if [ ! -z ${gitrepo+x} ]; then
@@ -64,8 +65,11 @@ function unpack {
fi
# FIXTHIS: support commit ids (log and checkout)
# FIXTHIS: strip components like tarballs
- git clone --depth=1 --branch=$gitbranch $gitrepo
+ git clone --depth=1 --branch=$gitbranch $gitrepo .
+ return
fi
+
+ echo "No tarball or gitrepo definition."
}
function is_empty {
@@ -191,7 +195,7 @@ function pre_build {
fi
if [ ! -e test_suite_ready ]; then
- unpack
+ unpack || abort_job "Error while unpacking"
return 0
fi
fi
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 08/19] main: remove ReBuild check
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (6 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 07/19] unpack: handle possible errors Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers Daniel Sangorrin
` (11 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
If rebuild is false, build is not called ever including the
first time when we are suppose to build the test.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/main.sh | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/engine/scripts/main.sh b/engine/scripts/main.sh
index 597049a..a09bb20 100644
--- a/engine/scripts/main.sh
+++ b/engine/scripts/main.sh
@@ -35,9 +35,7 @@ echo "##### doing fuego phase: pre_test ########"
pre_test
echo "##### doing fuego phase: build ########"
-if $Rebuild; then
- build
-fi
+build
echo "##### doing fuego phase: deploy ########"
deploy
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (7 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 08/19] main: remove ReBuild check Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 23:02 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 10/19] call_if_present: tests do not need to implement dummy functions Daniel Sangorrin
` (10 subsequent siblings)
19 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
This commit reimplements the way the "rebuild" flag is handled.
First, it unifies the variables 'build_failed' and 'test_suite_ready'
into a single 'fuego_test_succesfully_built' variable.
And second, the core code inside functions.sh is in charge of
setting this variable by reading the exit code from test_build.
Therefore, the main change is that now test developers only need
to care about the exit code returned by test_build. The variable
fuego_test_succesfully_built is handled by the core code, so
no need of redundant code on each test.
TODO: change the documentation for test creators.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 57 +++++++++++-----------
engine/tests/Benchmark.Dhrystone/fuego_test.sh | 6 +--
engine/tests/Benchmark.GLMark/fuego_test.sh | 2 +-
engine/tests/Benchmark.IOzone/fuego_test.sh | 4 +-
engine/tests/Benchmark.Interbench/fuego_test.sh | 2 +-
engine/tests/Benchmark.Java/fuego_test.sh | 4 --
engine/tests/Benchmark.Stream/fuego_test.sh | 2 +-
engine/tests/Benchmark.Whetstone/fuego_test.sh | 2 +-
engine/tests/Benchmark.aim7/fuego_test.sh | 2 +-
engine/tests/Benchmark.blobsallad/fuego_test.sh | 2 +-
engine/tests/Benchmark.bonnie/fuego_test.sh | 2 +-
engine/tests/Benchmark.cyclictest/fuego_test.sh | 2 +-
engine/tests/Benchmark.dbench/fuego_test.sh | 2 +-
engine/tests/Benchmark.ebizzy/fuego_test.sh | 2 +-
engine/tests/Benchmark.ffsb/fuego_test.sh | 2 +-
engine/tests/Benchmark.fio/fuego_test.sh | 1 -
.../Benchmark.fuego_check_plots/fuego_test.sh | 4 --
engine/tests/Benchmark.gtkperf/fuego_test.sh | 2 +-
engine/tests/Benchmark.hackbench/fuego_test.sh | 2 +-
engine/tests/Benchmark.himeno/fuego_test.sh | 2 +-
engine/tests/Benchmark.iperf/fuego_test.sh | 2 +-
engine/tests/Benchmark.linpack/fuego_test.sh | 2 +-
engine/tests/Benchmark.lmbench2/fuego_test.sh | 2 +-
engine/tests/Benchmark.nbench-byte/fuego_test.sh | 2 +-
engine/tests/Benchmark.nbench_byte/fuego_test.sh | 2 +-
engine/tests/Benchmark.netpipe/fuego_test.sh | 2 +-
engine/tests/Benchmark.signaltest/fuego_test.sh | 2 +-
engine/tests/Benchmark.tiobench/fuego_test.sh | 2 +-
engine/tests/Benchmark.x11perf/fuego_test.sh | 2 +-
engine/tests/Functional.LTP/fuego_test.sh | 2 -
engine/tests/Functional.aiostress/fuego_test.sh | 2 +-
engine/tests/Functional.bsdiff/fuego_test.sh | 4 --
engine/tests/Functional.bzip2/fuego_test.sh | 1 -
engine/tests/Functional.cmt/fuego_test.sh | 4 --
engine/tests/Functional.crashme/fuego_test.sh | 2 +-
engine/tests/Functional.curl/fuego_test.sh | 4 --
engine/tests/Functional.expat/fuego_test.sh | 4 +-
engine/tests/Functional.fixesproto/fuego_test.sh | 2 -
engine/tests/Functional.fuego_abort/fuego_test.sh | 4 --
.../Functional.fuego_board_check/fuego_test.sh | 4 --
.../Functional.fuego_test_phases/fuego_test.sh | 1 -
engine/tests/Functional.glib/fuego_test.sh | 2 +-
engine/tests/Functional.glibc/fuego_test.sh | 1 -
engine/tests/Functional.hello_world/fuego_test.sh | 2 +-
engine/tests/Functional.imagemagick/fuego_test.sh | 1 -
engine/tests/Functional.ipv6connect/fuego_test.sh | 2 +-
engine/tests/Functional.jpeg/fuego_test.sh | 1 -
engine/tests/Functional.libtar/fuego_test.sh | 1 -
engine/tests/Functional.linus_stress/fuego_test.sh | 2 +-
engine/tests/Functional.lwip/fuego_test.sh | 1 -
engine/tests/Functional.pi_tests/fuego_test.sh | 2 +-
engine/tests/Functional.pixman/fuego_test.sh | 1 -
engine/tests/Functional.pppd/fuego_test.sh | 1 -
engine/tests/Functional.protobuf/fuego_test.sh | 1 -
engine/tests/Functional.rmaptest/fuego_test.sh | 2 +-
engine/tests/Functional.scifab/fuego_test.sh | 4 --
engine/tests/Functional.scrashme/fuego_test.sh | 10 ++--
engine/tests/Functional.sdhi_0/fuego_test.sh | 4 --
engine/tests/Functional.stress/fuego_test.sh | 2 +-
engine/tests/Functional.synctest/fuego_test.sh | 2 +-
engine/tests/Functional.thrift/fuego_test.sh | 1 -
engine/tests/Functional.xorg-macros/fuego_test.sh | 1 -
engine/tests/Functional.zlib/fuego_test.sh | 2 +-
engine/tests/OpenSSL/openssl.sh | 1 -
engine/tests/netperf/netperf.sh | 2 +-
65 files changed, 75 insertions(+), 130 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index d5bf969..7ff77bc 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -159,12 +159,13 @@ function dump_syslogs {
ov_rootfs_logread "$@"
}
+# Wait if an instance of the same job (board-test-spec) is running
function concurrent_check {
LOCKFILE="$WORKSPACE/$TRIPLET.build.lock"
if [ -e ${LOCKFILE} ]; then
- while $(wget -qO- "$(cat ${LOCKFILE})/api/xml?xpath=*/building/text%28%29") && [ ! -e test_suite_ready ]
+ while $(wget -qO- "$(cat ${LOCKFILE})/api/xml?xpath=*/building/text%28%29") && [ ! -e fuego_test_succesfully_built ]
do
sleep 5
done
@@ -173,43 +174,41 @@ function concurrent_check {
echo "${BUILD_URL}" > ${LOCKFILE}
}
-function build_error () {
- touch build_failed
- abort_job "Build failed: $@"
-}
-
-# Wait for other builds of the same test running in parallel,
# process Rebuild flag, and unpack test sources if necessary.
-# Returns 0 if actual build needs to be performed; 1 - otherwise.
-# Build scripts must call this function in the beginning.
function pre_build {
- cd ${WORKSPACE}
+ cd ${WORKSPACE}
- mkdir -p $TRIPLET && cd $TRIPLET
- if concurrent_check; then
- [ "$Rebuild" = "true" ] && rm -rf *
+ mkdir -p $TRIPLET && cd $TRIPLET
- if [ -e build_failed ]; then
- rm $TRIPLET/build_failed
- rm -rf *
- fi
+ concurrent_check
- if [ ! -e test_suite_ready ]; then
- unpack || abort_job "Error while unpacking"
- return 0
+ if [ "$Rebuild" = "false" ] && [ -e fuego_test_succesfully_built ]; then
+ echo "The test is already built"
+ else
+ rm -rf *
+ unpack || abort_job "Error while unpacking"
fi
- fi
- return 1
}
function build {
- pre_build
- build_start_time=$(date +"%s.%N")
- test_build || return 1
- build_end_time=$(date +"%s.%N")
- build_duration=$(python -c "print $build_end_time - $build_start_time")
- echo "Fuego test_build duration=$build_duration seconds"
- post_build
+ local ret
+
+ pre_build
+ if [ -e fuego_test_succesfully_built ]; then
+ build_duration=0
+ else
+ build_start_time=$(date +"%s.%N")
+ call_if_present test_build && ret=0 || ret=$?
+ build_end_time=$(date +"%s.%N")
+ build_duration=$(python -c "print $build_end_time - $build_start_time")
+ if [ $ret -eq 0 ]; then
+ touch fuego_test_succesfully_built
+ else
+ abort_job "ERROR: test_build returned $ret"
+ fi
+ fi
+ echo "Fuego test_build duration=$build_duration seconds"
+ post_build
}
function post_build {
diff --git a/engine/tests/Benchmark.Dhrystone/fuego_test.sh b/engine/tests/Benchmark.Dhrystone/fuego_test.sh
index bc3f4c8..5f5f2ed 100755
--- a/engine/tests/Benchmark.Dhrystone/fuego_test.sh
+++ b/engine/tests/Benchmark.Dhrystone/fuego_test.sh
@@ -5,14 +5,14 @@ function test_pre_check {
}
function test_build {
- patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return 1
+ patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return
CFLAGS+=" -DTIME"
LDFLAGS+=" -lm"
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch test_suite_ready || return 1
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
}
function test_deploy {
- put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/ || return 1
+ put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
diff --git a/engine/tests/Benchmark.GLMark/fuego_test.sh b/engine/tests/Benchmark.GLMark/fuego_test.sh
index f409109..7deb6e2 100755
--- a/engine/tests/Benchmark.GLMark/fuego_test.sh
+++ b/engine/tests/Benchmark.GLMark/fuego_test.sh
@@ -7,7 +7,7 @@ function test_build {
-L${SDKROOT}/usr/lib -Wl,-rpath-link=${SDKROOT}/usr/lib \
-L${SDKROOT}/lib \
*.cpp -o glmark -lSDL -lGL \
- -lGLU -lGLEW && touch test_suite_ready || build_error "error while building test"
+ -lGLU -lGLEW
# -Wl,--allow-shlib-undefined *.cpp -o glmark -lSDL -lGL \
}
diff --git a/engine/tests/Benchmark.IOzone/fuego_test.sh b/engine/tests/Benchmark.IOzone/fuego_test.sh
index fe06e75..16b5c61 100755
--- a/engine/tests/Benchmark.IOzone/fuego_test.sh
+++ b/engine/tests/Benchmark.IOzone/fuego_test.sh
@@ -11,10 +11,10 @@ function test_build {
TARGET=linux-AMD64
else
echo "platform based on $ARCHITECTURE is not supported by benchmark"
- build_error "error while building test"
+ return 1
fi
- make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while building test"
+ make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.Interbench/fuego_test.sh b/engine/tests/Benchmark.Interbench/fuego_test.sh
index 5f950e2..31e6d2f 100755
--- a/engine/tests/Benchmark.Interbench/fuego_test.sh
+++ b/engine/tests/Benchmark.Interbench/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=interbench-0.31.tar.bz2
function test_build {
patch -p0 < $TEST_HOME/interbench.c.patch
- make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.Java/fuego_test.sh b/engine/tests/Benchmark.Java/fuego_test.sh
index 7978263..22cdbb7 100755
--- a/engine/tests/Benchmark.Java/fuego_test.sh
+++ b/engine/tests/Benchmark.Java/fuego_test.sh
@@ -1,9 +1,5 @@
tarball=java_perf.tar
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
put *.jar $BOARD_TESTDIR/fuego.$TESTDIR/
}
diff --git a/engine/tests/Benchmark.Stream/fuego_test.sh b/engine/tests/Benchmark.Stream/fuego_test.sh
index ddeb26c..9f4c04c 100755
--- a/engine/tests/Benchmark.Stream/fuego_test.sh
+++ b/engine/tests/Benchmark.Stream/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=stream.tar.bz2
function test_build {
- make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while building test"
+ make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.Whetstone/fuego_test.sh b/engine/tests/Benchmark.Whetstone/fuego_test.sh
index 925bfcb..5e791f0 100755
--- a/engine/tests/Benchmark.Whetstone/fuego_test.sh
+++ b/engine/tests/Benchmark.Whetstone/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=Whetstone.tar.bz2
function test_build {
CFLAGS+=" -DTIME"
- make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS" LIBS=" -lm" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS" LIBS=" -lm"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.aim7/fuego_test.sh b/engine/tests/Benchmark.aim7/fuego_test.sh
index 2cee552..87f9df9 100755
--- a/engine/tests/Benchmark.aim7/fuego_test.sh
+++ b/engine/tests/Benchmark.aim7/fuego_test.sh
@@ -3,7 +3,7 @@ tarball=osdl-aim-7.0.1.13.tar.gz
function test_build {
./bootstrap
PKG_CONFIG_PATH=${SDKROOT}/usr/lib/pkgconfig PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_SYSROOT_DIR=${SDKROOT} ./configure --host=$HOST --build=`uname -m`-linux-gnu LDFLAGS=-L${SDKROOT}/usr/lib CPPFLAGS=-I${SDKROOT}/usr/include CFLAGS=-I${SDKROOT}/usr/include LIBS=-laio --prefix=$BOARD_TESTDIR/$TESTDIR --datarootdir=$BOARD_TESTDIR/$TESTDIR
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Benchmark.blobsallad/fuego_test.sh b/engine/tests/Benchmark.blobsallad/fuego_test.sh
index ed5273d..f849946 100755
--- a/engine/tests/Benchmark.blobsallad/fuego_test.sh
+++ b/engine/tests/Benchmark.blobsallad/fuego_test.sh
@@ -5,7 +5,7 @@ function test_build {
patch -p0 -N -s < $TEST_HOME/blobsallad.auto.patch
patch -p0 -N -s < $TEST_HOME/bs_main.c.patch
- make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" SDKROOT="$SDKROOT" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" SDKROOT="$SDKROOT"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.bonnie/fuego_test.sh b/engine/tests/Benchmark.bonnie/fuego_test.sh
index 562088a..d94dce4 100755
--- a/engine/tests/Benchmark.bonnie/fuego_test.sh
+++ b/engine/tests/Benchmark.bonnie/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=bonnie++-1.03e.tar.gz
function test_build {
./configure --host=$HOST --build=`uname -m`-linux-gnu;
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Benchmark.cyclictest/fuego_test.sh b/engine/tests/Benchmark.cyclictest/fuego_test.sh
index 657de9b..2631a82 100755
--- a/engine/tests/Benchmark.cyclictest/fuego_test.sh
+++ b/engine/tests/Benchmark.cyclictest/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=cyclictest.tar.gz
function test_build {
- make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.dbench/fuego_test.sh b/engine/tests/Benchmark.dbench/fuego_test.sh
index d84c820..f7160f9 100755
--- a/engine/tests/Benchmark.dbench/fuego_test.sh
+++ b/engine/tests/Benchmark.dbench/fuego_test.sh
@@ -3,7 +3,7 @@ tarball=dbench-3.04.tar.gz
function test_build {
patch -N -s -p1 < $TEST_HOME/dbench_startup.patch
./configure --host=$HOST --build=`uname -m`-linux-gnu CFLAGS="$CFLAGS";
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Benchmark.ebizzy/fuego_test.sh b/engine/tests/Benchmark.ebizzy/fuego_test.sh
index 1e1e7ac..28b9fc0 100755
--- a/engine/tests/Benchmark.ebizzy/fuego_test.sh
+++ b/engine/tests/Benchmark.ebizzy/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=ebizzy-0.3.tar.gz
function test_build {
- $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c && touch test_suite_ready || build_error "error while building test"
+ $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c
}
function test_deploy {
diff --git a/engine/tests/Benchmark.ffsb/fuego_test.sh b/engine/tests/Benchmark.ffsb/fuego_test.sh
index 2037c8f..80a157d 100755
--- a/engine/tests/Benchmark.ffsb/fuego_test.sh
+++ b/engine/tests/Benchmark.ffsb/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=ffsb-6.0-rc2.tar.bz2
function test_build {
./configure --host=$HOST --build=`uname -m`-linux-gnu CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP CFLAGS="$CFLAGS";
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Benchmark.fio/fuego_test.sh b/engine/tests/Benchmark.fio/fuego_test.sh
index 344e445..039ec15 100755
--- a/engine/tests/Benchmark.fio/fuego_test.sh
+++ b/engine/tests/Benchmark.fio/fuego_test.sh
@@ -14,7 +14,6 @@ function test_build {
# >/dev/null
make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
- touch ../test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
index 562b659..80e4999 100644
--- a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
+++ b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
@@ -1,7 +1,3 @@
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
return 0
}
diff --git a/engine/tests/Benchmark.gtkperf/fuego_test.sh b/engine/tests/Benchmark.gtkperf/fuego_test.sh
index 07a4487..45b07bb 100755
--- a/engine/tests/Benchmark.gtkperf/fuego_test.sh
+++ b/engine/tests/Benchmark.gtkperf/fuego_test.sh
@@ -9,7 +9,7 @@ function test_build {
cd ..
export PATH=/usr/local/bin:$PATH
./configure --prefix=$BOARD_TESTDIR/$TESTDIR --host=$HOST --build=`uname -m`-linux-gnu --target=$HOST
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Benchmark.hackbench/fuego_test.sh b/engine/tests/Benchmark.hackbench/fuego_test.sh
index 3b45dd3..385f533 100755
--- a/engine/tests/Benchmark.hackbench/fuego_test.sh
+++ b/engine/tests/Benchmark.hackbench/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=hackbench.tar.gz
function test_build {
- $CC -lpthread hackbench.c -o hackbench && touch test_suite_ready || build_error "error while building test"
+ $CC -lpthread hackbench.c -o hackbench
}
function test_deploy {
diff --git a/engine/tests/Benchmark.himeno/fuego_test.sh b/engine/tests/Benchmark.himeno/fuego_test.sh
index debce2f..5ad2429 100755
--- a/engine/tests/Benchmark.himeno/fuego_test.sh
+++ b/engine/tests/Benchmark.himeno/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=himeno.tar.bz2
function test_build {
CFLAGS+=" -O3"
- make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.iperf/fuego_test.sh b/engine/tests/Benchmark.iperf/fuego_test.sh
index 74b4437..ad19052 100755
--- a/engine/tests/Benchmark.iperf/fuego_test.sh
+++ b/engine/tests/Benchmark.iperf/fuego_test.sh
@@ -5,7 +5,7 @@ function test_build {
make config.h
sed -i -e "s/#define HAVE_MALLOC 0/#define HAVE_MALLOC 1/g" -e "s/#define malloc rpl_malloc/\/\* #undef malloc \*\//g" config.h
sed -i -e '/HEADERS\(\)/ a\#include "gnu_getopt.h"' src/Settings.cpp
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Benchmark.linpack/fuego_test.sh b/engine/tests/Benchmark.linpack/fuego_test.sh
index a42d665..98ac835 100755
--- a/engine/tests/Benchmark.linpack/fuego_test.sh
+++ b/engine/tests/Benchmark.linpack/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=linpack.tar.bz2
function test_build {
patch -p0 -N -s < $TEST_HOME/linpack.c.patch
- $CC $CFLAGS -O -lm -o linpack linpack.c && touch test_suite_ready || build_error "error while building test"
+ $CC $CFLAGS -O -lm -o linpack linpack.c
}
function test_deploy {
diff --git a/engine/tests/Benchmark.lmbench2/fuego_test.sh b/engine/tests/Benchmark.lmbench2/fuego_test.sh
index 67d4afe..7dd56ca 100755
--- a/engine/tests/Benchmark.lmbench2/fuego_test.sh
+++ b/engine/tests/Benchmark.lmbench2/fuego_test.sh
@@ -11,7 +11,7 @@ function test_build {
patch -p0 < $TEST_HOME/bench.h.patch
cd ..
CFLAGS+=" -g -O"
- make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch test_suite_ready || build_error "error while building test"
+ make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.nbench-byte/fuego_test.sh b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
index 650b71f..d4a8336 100755
--- a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
+++ b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
@@ -4,7 +4,7 @@ function test_build {
patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
rm -f pointer.h && touch pointer.h
CFLAGS+=" -s -static -Wall -O3"
- make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.nbench_byte/fuego_test.sh b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
index 42b78d1..2561951 100755
--- a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
+++ b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
@@ -4,7 +4,7 @@ function test_build {
patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
rm -f pointer.h && touch pointer.h
CFLAGS+=" -s -static -Wall -O3"
- make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP
}
function test_deploy {
diff --git a/engine/tests/Benchmark.netpipe/fuego_test.sh b/engine/tests/Benchmark.netpipe/fuego_test.sh
index 732982e..a2f1a79 100755
--- a/engine/tests/Benchmark.netpipe/fuego_test.sh
+++ b/engine/tests/Benchmark.netpipe/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=NetPIPE-3.7.1.tar.gz
function test_build {
patch -p1 -N -s < ../../tarballs/netpipe-makefile.patch
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.signaltest/fuego_test.sh b/engine/tests/Benchmark.signaltest/fuego_test.sh
index 7c0e3fb..fc57c23 100755
--- a/engine/tests/Benchmark.signaltest/fuego_test.sh
+++ b/engine/tests/Benchmark.signaltest/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=signaltest.tar.gz
function test_build {
- make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.tiobench/fuego_test.sh b/engine/tests/Benchmark.tiobench/fuego_test.sh
index a974859..f916335 100755
--- a/engine/tests/Benchmark.tiobench/fuego_test.sh
+++ b/engine/tests/Benchmark.tiobench/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=tiobench-0.3.3.tar.gz
function test_build {
patch -N -s -p1 < $TEST_HOME/tiobench-fix-conflicting-types.patch
- make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}" && touch test_suite_ready || build_error "error while building test"
+ make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}"
}
function test_deploy {
diff --git a/engine/tests/Benchmark.x11perf/fuego_test.sh b/engine/tests/Benchmark.x11perf/fuego_test.sh
index b706d2b..70209c1 100755
--- a/engine/tests/Benchmark.x11perf/fuego_test.sh
+++ b/engine/tests/Benchmark.x11perf/fuego_test.sh
@@ -8,7 +8,7 @@ function test_build {
./configure $CONFIGURE_FLAGS X11PERF_LIBS="-lX11 -lXmu" # force to use libXmu instead of libXmuu
sed -i 's#$(SED)#sed#' Makefile # this is wrong, but somewhy $(SED) in Makefile expands to nothing
- make && touch test_suite_ready
+ make
}
function test_deploy {
diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index db5c4d8..f7c9ced 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -89,8 +89,6 @@ function test_build {
cp --parents testcases/realtime/scripts/setenv.sh `pwd`/target_bin/
cp $TEST_HOME/ltp_target_run.sh `pwd`/target_bin/
-
- touch test_suite_ready
else
echo "Skipping LTP build"
fi
diff --git a/engine/tests/Functional.aiostress/fuego_test.sh b/engine/tests/Functional.aiostress/fuego_test.sh
index 2bebb67..7418af9 100755
--- a/engine/tests/Functional.aiostress/fuego_test.sh
+++ b/engine/tests/Functional.aiostress/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=aiostress.tar.gz
function test_build {
- $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio aiostress.c -o aiostress && touch test_suite_ready || build_error "error while building test"
+ $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio aiostress.c -o aiostress
}
function test_deploy {
diff --git a/engine/tests/Functional.bsdiff/fuego_test.sh b/engine/tests/Functional.bsdiff/fuego_test.sh
index 48723c0..b84c259 100755
--- a/engine/tests/Functional.bsdiff/fuego_test.sh
+++ b/engine/tests/Functional.bsdiff/fuego_test.sh
@@ -1,9 +1,5 @@
tarball=bsdiff.tar.gz
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
put data/*.modified data/*.original $BOARD_TESTDIR/fuego.$TESTDIR/;
}
diff --git a/engine/tests/Functional.bzip2/fuego_test.sh b/engine/tests/Functional.bzip2/fuego_test.sh
index f5c75c5..2375b48 100755
--- a/engine/tests/Functional.bzip2/fuego_test.sh
+++ b/engine/tests/Functional.bzip2/fuego_test.sh
@@ -22,7 +22,6 @@ function test_build {
if cmp sample3.bz2 sample3.rb2; then echo 'TEST-9 OK'; else echo 'TEST-9 FAILED'; fi;
if cmp sample1.tst sample1.ref; then echo 'TEST-10 OK'; else echo 'TEST-10 FAILED'; fi;
if cmp sample2.tst sample2.ref; then echo 'TEST-11 OK'; else echo 'TEST-11 FAILED'; fi;" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.cmt/fuego_test.sh b/engine/tests/Functional.cmt/fuego_test.sh
index 5a0482d..fc662b3 100755
--- a/engine/tests/Functional.cmt/fuego_test.sh
+++ b/engine/tests/Functional.cmt/fuego_test.sh
@@ -1,9 +1,5 @@
tarball=dung-3.4.25-m2.tar.gz
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
put ./* $BOARD_TESTDIR/fuego.$TESTDIR/
}
diff --git a/engine/tests/Functional.crashme/fuego_test.sh b/engine/tests/Functional.crashme/fuego_test.sh
index f0aea34..66fa374 100755
--- a/engine/tests/Functional.crashme/fuego_test.sh
+++ b/engine/tests/Functional.crashme/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=crashme_2.4.tar.bz2
function test_build {
patch -p1 -N -s < $TEST_HOME/crashme_2.4-9.patch
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
}
function test_deploy {
diff --git a/engine/tests/Functional.curl/fuego_test.sh b/engine/tests/Functional.curl/fuego_test.sh
index 945e0e2..81059d8 100755
--- a/engine/tests/Functional.curl/fuego_test.sh
+++ b/engine/tests/Functional.curl/fuego_test.sh
@@ -1,7 +1,3 @@
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
true
}
diff --git a/engine/tests/Functional.expat/fuego_test.sh b/engine/tests/Functional.expat/fuego_test.sh
index 5a0f4f5..8a84d89 100755
--- a/engine/tests/Functional.expat/fuego_test.sh
+++ b/engine/tests/Functional.expat/fuego_test.sh
@@ -14,8 +14,8 @@ function test_build {
CXXFLAGS='-I. -I./lib -g -O2'
./configure --build=`uname -m`-gnu-linux --host="$PREFIX" #CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
- make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" 1>/dev/null
- make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX" CXXFLAGS="$CXXFLAGS" tests/runtestspp; 1>/dev/null && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" || return
+ make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX" CXXFLAGS="$CXXFLAGS" tests/runtestspp
}
function test_deploy {
diff --git a/engine/tests/Functional.fixesproto/fuego_test.sh b/engine/tests/Functional.fixesproto/fuego_test.sh
index 86919b9..cebd40c 100755
--- a/engine/tests/Functional.fixesproto/fuego_test.sh
+++ b/engine/tests/Functional.fixesproto/fuego_test.sh
@@ -6,8 +6,6 @@ function test_build {
if [ -f /usr/include/X11/extensions/xfixeswire.h ]; then echo 'TEST-2 OK'; else echo 'TEST-2 FAIL'; fi;
if [ -f /usr/lib/pkgconfig/fixesproto.pc ]; then echo 'TEST-3 OK'; else echo 'TEST-3 FAIL'; fi;
" > run-tests.sh
-
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.fuego_abort/fuego_test.sh b/engine/tests/Functional.fuego_abort/fuego_test.sh
index b7cab8e..84f0a42 100755
--- a/engine/tests/Functional.fuego_abort/fuego_test.sh
+++ b/engine/tests/Functional.fuego_abort/fuego_test.sh
@@ -9,10 +9,6 @@ function test_pre_check {
echo "This test has pid $$"
}
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
true
}
diff --git a/engine/tests/Functional.fuego_board_check/fuego_test.sh b/engine/tests/Functional.fuego_board_check/fuego_test.sh
index 580d623..972c46e 100755
--- a/engine/tests/Functional.fuego_board_check/fuego_test.sh
+++ b/engine/tests/Functional.fuego_board_check/fuego_test.sh
@@ -2,10 +2,6 @@
tarfile=scan_for_items.sh
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
put $TEST_HOME/scan_for_items.sh $BOARD_TESTDIR/fuego.$TESTDIR/
}
diff --git a/engine/tests/Functional.fuego_test_phases/fuego_test.sh b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
index 80251c9..8834a3a 100755
--- a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
+++ b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
@@ -14,7 +14,6 @@ function test_pre_check {
function test_build {
PHASE_STR="${PHASE_STR}:test_build"
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.glib/fuego_test.sh b/engine/tests/Functional.glib/fuego_test.sh
index c346f05..cbeb35f 100755
--- a/engine/tests/Functional.glib/fuego_test.sh
+++ b/engine/tests/Functional.glib/fuego_test.sh
@@ -8,7 +8,7 @@ function test_build {
./configure --prefix=`pwd`/build --cache-file=config.cross --host=$HOST --target=$HOST --build=`uname -m`-linux-gnu CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CXXFLAGS="$CXXFLAGS"
make 2>&1
cd tests && make test; cd -
- cd glib/tests && make test; cd - && touch test_suite_ready || build_error "error while building test"
+ cd glib/tests && make test; cd -
}
function test_deploy {
diff --git a/engine/tests/Functional.glibc/fuego_test.sh b/engine/tests/Functional.glibc/fuego_test.sh
index e387900..6e3213c 100755
--- a/engine/tests/Functional.glibc/fuego_test.sh
+++ b/engine/tests/Functional.glibc/fuego_test.sh
@@ -39,7 +39,6 @@ EOF
then echo 'TEST-14 OK'; else echo 'TEST-14 FAIL'; fi;
" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.hello_world/fuego_test.sh b/engine/tests/Functional.hello_world/fuego_test.sh
index 27a8344..9c9a394 100755
--- a/engine/tests/Functional.hello_world/fuego_test.sh
+++ b/engine/tests/Functional.hello_world/fuego_test.sh
@@ -3,7 +3,7 @@
tarball=hello-test-1.1.tgz
function test_build {
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Functional.imagemagick/fuego_test.sh b/engine/tests/Functional.imagemagick/fuego_test.sh
index 3f1c69f..b21eb77 100755
--- a/engine/tests/Functional.imagemagick/fuego_test.sh
+++ b/engine/tests/Functional.imagemagick/fuego_test.sh
@@ -44,7 +44,6 @@ function test_build {
# if ./djpeg -dct int -ppm -outfile testout.ppm testorig.jpg; then echo 'TEST-1 OK'; else echo 'TEST-1 FAIL'; fi;
# " > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.ipv6connect/fuego_test.sh b/engine/tests/Functional.ipv6connect/fuego_test.sh
index bcda1f3..c6fdd05 100755
--- a/engine/tests/Functional.ipv6connect/fuego_test.sh
+++ b/engine/tests/Functional.ipv6connect/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=ipv6connect.tar.gz
function test_build {
- make CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" LD="$LD"
}
function test_deploy {
diff --git a/engine/tests/Functional.jpeg/fuego_test.sh b/engine/tests/Functional.jpeg/fuego_test.sh
index 55f34d6..2cbc237 100755
--- a/engine/tests/Functional.jpeg/fuego_test.sh
+++ b/engine/tests/Functional.jpeg/fuego_test.sh
@@ -16,7 +16,6 @@ function test_build {
if cmp testimg.ppm testoutp.ppm; then echo 'TEST-10 OK'; else echo 'TEST-10 FAIL'; fi;
if cmp testimgp.jpg testoutp.jpg; then echo 'TEST-11 OK'; else echo 'TEST-11 FAIL'; fi;
if cmp testorig.jpg testoutt.jpg; then echo 'TEST-12 OK'; else echo 'TEST-12 FAIL'; fi;" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.libtar/fuego_test.sh b/engine/tests/Functional.libtar/fuego_test.sh
index 06ef34b..2fcc01f 100755
--- a/engine/tests/Functional.libtar/fuego_test.sh
+++ b/engine/tests/Functional.libtar/fuego_test.sh
@@ -22,7 +22,6 @@ function test_build {
else
echo 'TEST-3 FAILED'
fi" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.linus_stress/fuego_test.sh b/engine/tests/Functional.linus_stress/fuego_test.sh
index b5b2afb..a03debd 100755
--- a/engine/tests/Functional.linus_stress/fuego_test.sh
+++ b/engine/tests/Functional.linus_stress/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=linus_stress.tar.gz
function test_build {
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
}
function test_deploy {
diff --git a/engine/tests/Functional.lwip/fuego_test.sh b/engine/tests/Functional.lwip/fuego_test.sh
index d5c8c80..17d15af 100755
--- a/engine/tests/Functional.lwip/fuego_test.sh
+++ b/engine/tests/Functional.lwip/fuego_test.sh
@@ -29,7 +29,6 @@ function test_build {
killall sntp_test
" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.pi_tests/fuego_test.sh b/engine/tests/Functional.pi_tests/fuego_test.sh
index de6057e..c3240cc 100755
--- a/engine/tests/Functional.pi_tests/fuego_test.sh
+++ b/engine/tests/Functional.pi_tests/fuego_test.sh
@@ -2,7 +2,7 @@ gitrepo=https://github.com/clrkwllms/rt-tests.git
function test_build {
cd rt-tests/
- make CC="$CC" LD="$LD" NUMA=0 pi_stress && touch test_suite_ready || build_error "error while building test"
+ make CC="$CC" LD="$LD" NUMA=0 pi_stress
}
function test_deploy {
diff --git a/engine/tests/Functional.pixman/fuego_test.sh b/engine/tests/Functional.pixman/fuego_test.sh
index 7999b0d..a8cd6dd 100755
--- a/engine/tests/Functional.pixman/fuego_test.sh
+++ b/engine/tests/Functional.pixman/fuego_test.sh
@@ -190,7 +190,6 @@ function test_build {
echo 'TEST-30 FAILED'
fi
" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.pppd/fuego_test.sh b/engine/tests/Functional.pppd/fuego_test.sh
index a622f95..d080871 100755
--- a/engine/tests/Functional.pppd/fuego_test.sh
+++ b/engine/tests/Functional.pppd/fuego_test.sh
@@ -22,7 +22,6 @@ function test_build {
killall ppp_response
# killall pppd
" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.protobuf/fuego_test.sh b/engine/tests/Functional.protobuf/fuego_test.sh
index 57037d3..2c6cfcb 100755
--- a/engine/tests/Functional.protobuf/fuego_test.sh
+++ b/engine/tests/Functional.protobuf/fuego_test.sh
@@ -27,7 +27,6 @@ function test_build {
else
echo 'TEST-4 FAILED'
fi" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.rmaptest/fuego_test.sh b/engine/tests/Functional.rmaptest/fuego_test.sh
index 69e08b0..84e360b 100755
--- a/engine/tests/Functional.rmaptest/fuego_test.sh
+++ b/engine/tests/Functional.rmaptest/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=rmaptest.tar.gz
function test_build {
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
}
function test_deploy {
diff --git a/engine/tests/Functional.scifab/fuego_test.sh b/engine/tests/Functional.scifab/fuego_test.sh
index d874964..418e947 100755
--- a/engine/tests/Functional.scifab/fuego_test.sh
+++ b/engine/tests/Functional.scifab/fuego_test.sh
@@ -1,9 +1,5 @@
tarball=dung-3.4.25-m2.tar.gz
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
put ./* $OSV_HOME/osv.$TESTDIR/
}
diff --git a/engine/tests/Functional.scrashme/fuego_test.sh b/engine/tests/Functional.scrashme/fuego_test.sh
index d173e47..aabb779 100755
--- a/engine/tests/Functional.scrashme/fuego_test.sh
+++ b/engine/tests/Functional.scrashme/fuego_test.sh
@@ -1,19 +1,19 @@
tarball=scrashme.tar.bz2
function test_pre_check {
- assert_define FUNCTIONAL_SCRASHME_NUM
- assert_define FUNCTIONAL_SCRASHME_MODE
+ assert_define FUNCTIONAL_SCRASHME_NUM
+ assert_define FUNCTIONAL_SCRASHME_MODE
}
function test_build {
patch -p1 -N -s < $TEST_HOME/scrashme-testfix.patch
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
}
function test_deploy {
- put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
+ put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --mode=$FUNCTIONAL_SCRASHME_MODE -N$FUNCTIONAL_SCRASHME_NUM"
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --mode=$FUNCTIONAL_SCRASHME_MODE -N$FUNCTIONAL_SCRASHME_NUM"
}
diff --git a/engine/tests/Functional.sdhi_0/fuego_test.sh b/engine/tests/Functional.sdhi_0/fuego_test.sh
index b290808..b3cee43 100755
--- a/engine/tests/Functional.sdhi_0/fuego_test.sh
+++ b/engine/tests/Functional.sdhi_0/fuego_test.sh
@@ -1,9 +1,5 @@
tarball=dung-3.4.25-m2.tar.gz
-function test_build {
- touch test_suite_ready
-}
-
function test_deploy {
put ./* $OSV_HOME/osv.$TESTDIR/
}
diff --git a/engine/tests/Functional.stress/fuego_test.sh b/engine/tests/Functional.stress/fuego_test.sh
index e9c467f..11bcfe1 100755
--- a/engine/tests/Functional.stress/fuego_test.sh
+++ b/engine/tests/Functional.stress/fuego_test.sh
@@ -2,7 +2,7 @@ tarball=stress-1.0.4.tar.gz
function test_build {
./configure --host=$HOST --build=`uname -m`-linux-gnu CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS";
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
function test_deploy {
diff --git a/engine/tests/Functional.synctest/fuego_test.sh b/engine/tests/Functional.synctest/fuego_test.sh
index aff0820..95d4dda 100755
--- a/engine/tests/Functional.synctest/fuego_test.sh
+++ b/engine/tests/Functional.synctest/fuego_test.sh
@@ -1,7 +1,7 @@
tarball=synctest.tar.gz
function test_build {
- make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" && touch test_suite_ready || build_error "error while building test"
+ make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
}
function test_deploy {
diff --git a/engine/tests/Functional.thrift/fuego_test.sh b/engine/tests/Functional.thrift/fuego_test.sh
index 8310e4e..17a8c11 100755
--- a/engine/tests/Functional.thrift/fuego_test.sh
+++ b/engine/tests/Functional.thrift/fuego_test.sh
@@ -111,7 +111,6 @@ function test_build {
killall CppServer
" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.xorg-macros/fuego_test.sh b/engine/tests/Functional.xorg-macros/fuego_test.sh
index 95490be..df10fcd 100755
--- a/engine/tests/Functional.xorg-macros/fuego_test.sh
+++ b/engine/tests/Functional.xorg-macros/fuego_test.sh
@@ -6,7 +6,6 @@ function test_build {
if [ -f /usr/share/pkgconfig/xorg-macros.pc ]; then echo 'TEST-2 OK'; else echo 'TEST-2 FAIL'; fi;
" > run-tests.sh
- touch test_suite_ready
}
function test_deploy {
diff --git a/engine/tests/Functional.zlib/fuego_test.sh b/engine/tests/Functional.zlib/fuego_test.sh
index 7615b5d..c794c52 100755
--- a/engine/tests/Functional.zlib/fuego_test.sh
+++ b/engine/tests/Functional.zlib/fuego_test.sh
@@ -4,7 +4,7 @@ function test_build {
AR=$PREFIX'-ar rc'
CPP=$PREFIX'-gcc -E'
CC="$CC" AR="$AR" CPP="$CPP" ./configure --includedir=$SDKROOT/usr/include --libdir=$SDKROOT/usr/lib
- make LDSHARED="$CC" >/dev/null && touch test_suite_ready || build_error "error while building test"
+ make LDSHARED="$CC" >/dev/null
}
function test_deploy {
diff --git a/engine/tests/OpenSSL/openssl.sh b/engine/tests/OpenSSL/openssl.sh
index 95848a7..ec062c7 100755
--- a/engine/tests/OpenSSL/openssl.sh
+++ b/engine/tests/OpenSSL/openssl.sh
@@ -51,5 +51,4 @@ function test_build {
../util/shlib_wrap.sh ./shatest
../util/shlib_wrap.sh ./ssltest' > run-tests.sh
rm test/fips_aes_data
- touch test_suite_ready || build_error "error while building test"
}
diff --git a/engine/tests/netperf/netperf.sh b/engine/tests/netperf/netperf.sh
index 2cd6da4..8c14cff 100755
--- a/engine/tests/netperf/netperf.sh
+++ b/engine/tests/netperf/netperf.sh
@@ -1,6 +1,6 @@
function test_build {
echo "ac_cv_func_setpgrp_void=yes" > config.cache
./configure --build=`./config.guess` --host=$HOST CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" --config-cache
- make && touch test_suite_ready || build_error "error while building test"
+ make
}
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 10/19] call_if_present: tests do not need to implement dummy functions
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (8 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 11/19] expat: remove unneeded semicolons Daniel Sangorrin
` (9 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Since we use the call_if_present macro, tests do not need
to implement dummy versions of the test_xxx functions.
TODO: add to the documentation a template containing all functions
so that test developers can start with that.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 2 +-
.../Benchmark.fuego_check_plots/fuego_test.sh | 4 ----
engine/tests/Benchmark.reboot/fuego_test.sh | 4 ----
engine/tests/Functional.aiostress/fuego_test.sh | 7 -------
engine/tests/Functional.curl/fuego_test.sh | 4 ----
engine/tests/Functional.fuego_abort/fuego_test.sh | 8 --------
.../tests/Functional.fuego_transport/fuego_test.sh | 22 ++++++----------------
engine/tests/Functional.hciattach/fuego_test.sh | 10 ----------
engine/tests/Functional.iptables/fuego_test.sh | 12 ++----------
engine/tests/Functional.ipv6connect/fuego_test.sh | 8 +-------
engine/tests/Functional.linus_stress/fuego_test.sh | 6 ------
engine/tests/Functional.net-tools/fuego_test.sh | 8 --------
engine/tests/Functional.rmaptest/fuego_test.sh | 18 ++++++------------
13 files changed, 16 insertions(+), 97 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index 7ff77bc..188408d 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -219,7 +219,7 @@ function deploy {
echo "##### doing fuego phase: pre_deploy ########"
pre_deploy
echo "##### doing fuego phase: test_deploy ########"
- test_deploy
+ call_if_present test_deploy
echo "##### doing fuego phase: post_deploy ########"
post_deploy
}
diff --git a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
index 80e4999..14fb3a2 100644
--- a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
+++ b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
@@ -1,7 +1,3 @@
-function test_deploy {
- return 0
-}
-
function test_run {
RESULT=$(( 1000 + (RANDOM % 600 - 300) ))
report "echo fuego_check_plots result: $RESULT"
diff --git a/engine/tests/Benchmark.reboot/fuego_test.sh b/engine/tests/Benchmark.reboot/fuego_test.sh
index 603e545..102c1de 100755
--- a/engine/tests/Benchmark.reboot/fuego_test.sh
+++ b/engine/tests/Benchmark.reboot/fuego_test.sh
@@ -1,9 +1,5 @@
tarball=none
-function test_build {
- true
-}
-
function test_deploy {
put $TEST_HOME/get_reboot_time.sh $BOARD_TESTDIR/fuego.$TESTDIR/
}
diff --git a/engine/tests/Functional.aiostress/fuego_test.sh b/engine/tests/Functional.aiostress/fuego_test.sh
index 7418af9..0e48c6b 100755
--- a/engine/tests/Functional.aiostress/fuego_test.sh
+++ b/engine/tests/Functional.aiostress/fuego_test.sh
@@ -18,10 +18,3 @@ function test_run {
hd_test_clean_umount $FUNCTIONAL_AIOSTRESS_MOUNT_BLOCKDEV $FUNCTIONAL_AIOSTRESS_MOUNT_POINT
}
-
-function test_processing {
- true
-}
-
-
-
diff --git a/engine/tests/Functional.curl/fuego_test.sh b/engine/tests/Functional.curl/fuego_test.sh
index 81059d8..90d24dd 100755
--- a/engine/tests/Functional.curl/fuego_test.sh
+++ b/engine/tests/Functional.curl/fuego_test.sh
@@ -1,7 +1,3 @@
-function test_deploy {
- true
-}
-
function test_run {
report "if curl -o test.html www.baidu.com > /dev/null; then echo 'TEST-1 OK'; else echo 'TEST-1 FAILED'; fi;\
if curl -i -r 0-1024 http://www.sina.com.cn -o sina_part1.html > /dev/null; then echo 'TEST-2 OK'; else echo 'TEST-2 FAILED'; fi;\
diff --git a/engine/tests/Functional.fuego_abort/fuego_test.sh b/engine/tests/Functional.fuego_abort/fuego_test.sh
index 84f0a42..1d108cd 100755
--- a/engine/tests/Functional.fuego_abort/fuego_test.sh
+++ b/engine/tests/Functional.fuego_abort/fuego_test.sh
@@ -9,10 +9,6 @@ function test_pre_check {
echo "This test has pid $$"
}
-function test_deploy {
- true
-}
-
function test_run {
# leave a process running on the target
cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; ln -sf /bin/sleep fuegosleep ; $PROGRAM_NOHUP ./fuegosleep 10000 >/dev/null 2>&1 &"
@@ -26,10 +22,6 @@ function test_run {
abort_job
}
-function test_processing {
- true
-}
-
function more_handler1 {
echo "in more_handler1 - got another SIGTERM"
echo "in more_handler1 - got another SIGTERM" >>$LOGFILE
diff --git a/engine/tests/Functional.fuego_transport/fuego_test.sh b/engine/tests/Functional.fuego_transport/fuego_test.sh
index bf4667b..c182e61 100644
--- a/engine/tests/Functional.fuego_transport/fuego_test.sh
+++ b/engine/tests/Functional.fuego_transport/fuego_test.sh
@@ -2,48 +2,40 @@
tarball=fuego-transport-1.0.tgz
-function test_build {
- return 0
-}
-
-function test_deploy {
- return 0
-}
-
function test_run {
# copy some files to the target under different conditions
# get is tested by the system, with log retrieval
desc="test single file put"
DESTDIR=$BOARD_TESTDIR/fuego.$TESTDIR
put bartop $DESTDIR
- report "if [ -f $DESTDIR/bartop ] ; then echo \"ok 1 $desc\" ; else echo \"not ok 1 $desc\" ; fi"
+ report "if [ -f $DESTDIR/bartop ] ; then echo \"ok 1 $desc\" ; else echo \"not ok 1 $desc\" ; fi"
cmd "rm $DESTDIR/bartop"
desc="test multiple file put"
put footop foo2top $DESTDIR
- report_append "if [ -f $DESTDIR/footop -a -f $DESTDIR/foo2top ] ; then echo \"ok 2 $desc\" ; else echo \"not ok 2 $desc\" ; fi"
+ report_append "if [ -f $DESTDIR/footop -a -f $DESTDIR/foo2top ] ; then echo \"ok 2 $desc\" ; else echo \"not ok 2 $desc\" ; fi"
cmd "rm $DESTDIR/foo*top"
desc="test wildcard file put"
put foo*top $DESTDIR
- report_append "if [ -f $DESTDIR/footop -a -f $DESTDIR/foo2top ] ; then echo \"ok 3 $desc\" ; else echo \"not ok 3 $desc\" ; fi"
+ report_append "if [ -f $DESTDIR/footop -a -f $DESTDIR/foo2top ] ; then echo \"ok 3 $desc\" ; else echo \"not ok 3 $desc\" ; fi"
cmd "rm $DESTDIR/foo*top"
desc="test recursive directory put"
put dir1 $DESTDIR
- report_append "if [ -f $DESTDIR/dir1/d1foo -a -f $DESTDIR/dir1/d1bar ] ; then echo \"ok 4 $desc\" ; else echo \"not ok 4 $desc\" ; fi"
+ report_append "if [ -f $DESTDIR/dir1/d1foo -a -f $DESTDIR/dir1/d1bar ] ; then echo \"ok 4 $desc\" ; else echo \"not ok 4 $desc\" ; fi"
cmd "rm $DESTDIR/dir1/*"
cmd "rmdir $DESTDIR/dir1"
desc="test multiple recursive dir put"
put dir1 dir2 $DESTDIR
- report_append "if [ -f $DESTDIR/dir1/d1foo -a -f $DESTDIR/dir1/d1bar -a -f $DESTDIR/dir2/d2foo -a -f $DESTDIR/dir2/d2bar ] ; then echo \"ok 5 $desc\" ; else echo \"not ok 5 $desc\" ; fi"
+ report_append "if [ -f $DESTDIR/dir1/d1foo -a -f $DESTDIR/dir1/d1bar -a -f $DESTDIR/dir2/d2foo -a -f $DESTDIR/dir2/d2bar ] ; then echo \"ok 5 $desc\" ; else echo \"not ok 5 $desc\" ; fi"
cmd "rm $DESTDIR/dir1/* $DESTDIR/dir2/*"
cmd "rmdir $DESTDIR/dir1 $DESTDIR/dir2"
desc="test multiple files, wildcards and recursive dirs put"
put bartop foo*top dir* $DESTDIR
- report_append "if [ -f $DESTDIR/bartop -a -f $DESTDIR/footop -a -f $DESTDIR/foo2top -a -f $DESTDIR/dir1/d1foo -a -f $DESTDIR/dir1/d1bar -a -f $DESTDIR/dir2/d2foo -a -f $DESTDIR/dir2/d2bar ] ; then echo \"ok 6 $desc\" ; else echo \"not ok 6 $desc\" ; fi"
+ report_append "if [ -f $DESTDIR/bartop -a -f $DESTDIR/footop -a -f $DESTDIR/foo2top -a -f $DESTDIR/dir1/d1foo -a -f $DESTDIR/dir1/d1bar -a -f $DESTDIR/dir2/d2foo -a -f $DESTDIR/dir2/d2bar ] ; then echo \"ok 6 $desc\" ; else echo \"not ok 6 $desc\" ; fi"
cmd "rm $DESTDIR/bartop $DESTDIR/foo*top $DESTDIR/dir1/* $DESTDIR/dir2/*"
cmd "rmdir $DESTDIR/dir1 $DESTDIR/dir2"
@@ -53,5 +45,3 @@ function test_run {
function test_processing {
log_compare "$TESTDIR" "6" "^ok" "p"
}
-
-
diff --git a/engine/tests/Functional.hciattach/fuego_test.sh b/engine/tests/Functional.hciattach/fuego_test.sh
index afe66aa..0082868 100755
--- a/engine/tests/Functional.hciattach/fuego_test.sh
+++ b/engine/tests/Functional.hciattach/fuego_test.sh
@@ -1,11 +1,3 @@
-function test_build {
- echo True
-}
-
-function test_deploy {
- echo True
-}
-
function test_run {
report "if hciattach -l;then echo 'TEST-1 OK'; else echo 'TEST-1 FAILED'; fi"
}
@@ -14,5 +6,3 @@ function test_processing {
log_compare "$TESTDIR" "1" "^TEST.*OK" "p"
log_compare "$TESTDIR" "0" "^TEST.*FAILED" "n"
}
-
-
diff --git a/engine/tests/Functional.iptables/fuego_test.sh b/engine/tests/Functional.iptables/fuego_test.sh
index 023c2d9..92a494a 100755
--- a/engine/tests/Functional.iptables/fuego_test.sh
+++ b/engine/tests/Functional.iptables/fuego_test.sh
@@ -1,11 +1,3 @@
-function test_build {
- echo ' build OK '
-}
-
-function test_deploy {
- echo ' deploy OK '
-}
-
function test_run {
report " \
if iptables -A INPUT -t filter -p tcp --dport 80 -j DROP; then \
@@ -18,7 +10,7 @@ function test_run {
else \
echo 'TEST-2 FAIL'; \
fi; \
- if iptables -t raw -I PREROUTING -p udp --dport 53 -j NOTRACK &&
+ if iptables -t raw -I PREROUTING -p udp --dport 53 -j NOTRACK &&
iptables -t raw -I OUTPUT -p udp --dport 53 -j NOTRACK; then \
echo 'TEST-3 OK'; \
else \
@@ -56,7 +48,7 @@ function test_run {
else \
echo 'TEST-9 FAIL'; \
fi \
- "
+ "
}
function test_processing {
diff --git a/engine/tests/Functional.ipv6connect/fuego_test.sh b/engine/tests/Functional.ipv6connect/fuego_test.sh
index c6fdd05..22454e9 100755
--- a/engine/tests/Functional.ipv6connect/fuego_test.sh
+++ b/engine/tests/Functional.ipv6connect/fuego_test.sh
@@ -9,11 +9,5 @@ function test_deploy {
}
function test_run {
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ipv6connect"
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ipv6connect"
}
-
-function test_processing {
- true
-}
-
-
diff --git a/engine/tests/Functional.linus_stress/fuego_test.sh b/engine/tests/Functional.linus_stress/fuego_test.sh
index a03debd..15265c4 100755
--- a/engine/tests/Functional.linus_stress/fuego_test.sh
+++ b/engine/tests/Functional.linus_stress/fuego_test.sh
@@ -21,9 +21,3 @@ function test_run {
cat ./dirty_ratio > /proc/sys/vm/dirty_ratio; \
cat ./dirty_bg > /proc/sys/vm/dirty_background_ratio || echo 'WARNING: /proc/sys/vm/dirty_background_ratio is read-only'"
}
-
-function test_processing {
- true
-}
-
-
diff --git a/engine/tests/Functional.net-tools/fuego_test.sh b/engine/tests/Functional.net-tools/fuego_test.sh
index b25f8b7..06c80b2 100755
--- a/engine/tests/Functional.net-tools/fuego_test.sh
+++ b/engine/tests/Functional.net-tools/fuego_test.sh
@@ -1,13 +1,5 @@
#!/bin/bash
-function test_build {
- true
-}
-
-function test_deploy {
- pwd
-}
-
function test_run {
report "if arp; then echo 'TEST-1 OK'; else echo 'TEST-1 FAIL'; fi; \
if hostname; then echo 'TEST-2 OK'; else echo 'TEST-2 FAIL'; fi; \
diff --git a/engine/tests/Functional.rmaptest/fuego_test.sh b/engine/tests/Functional.rmaptest/fuego_test.sh
index 84e360b..7dfeb69 100755
--- a/engine/tests/Functional.rmaptest/fuego_test.sh
+++ b/engine/tests/Functional.rmaptest/fuego_test.sh
@@ -9,17 +9,11 @@ function test_deploy {
}
function test_run {
- assert_define FUNCTIONAL_RMAPTEST_ITERATIONS
- assert_define FUNCTIONAL_RMAPTEST_VMA
- assert_define FUNCTIONAL_RMAPTEST_VMA_SIZE
- assert_define FUNCTIONAL_RMAPTEST_TASKS
- assert_define FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS
+ assert_define FUNCTIONAL_RMAPTEST_ITERATIONS
+ assert_define FUNCTIONAL_RMAPTEST_VMA
+ assert_define FUNCTIONAL_RMAPTEST_VMA_SIZE
+ assert_define FUNCTIONAL_RMAPTEST_TASKS
+ assert_define FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./rmap-test -h -i$FUNCTIONAL_RMAPTEST_ITERATIONS -n$FUNCTIONAL_RMAPTEST_VMA -s$FUNCTIONAL_RMAPTEST_VMA_SIZE -t$FUNCTIONAL_RMAPTEST_TASKS -V$FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS -v file1.dat"
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./rmap-test -h -i$FUNCTIONAL_RMAPTEST_ITERATIONS -n$FUNCTIONAL_RMAPTEST_VMA -s$FUNCTIONAL_RMAPTEST_VMA_SIZE -t$FUNCTIONAL_RMAPTEST_TASKS -V$FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS -v file1.dat"
}
-
-function test_processing {
- true
-}
-
-
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 11/19] expat: remove unneeded semicolons
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (9 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 10/19] call_if_present: tests do not need to implement dummy functions Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 12/19] tarball: remove unneeded tarball definitions Daniel Sangorrin
` (8 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Functional.expat/fuego_test.sh | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/engine/tests/Functional.expat/fuego_test.sh b/engine/tests/Functional.expat/fuego_test.sh
index 8a84d89..09b21ee 100755
--- a/engine/tests/Functional.expat/fuego_test.sh
+++ b/engine/tests/Functional.expat/fuego_test.sh
@@ -20,12 +20,12 @@ function test_build {
function test_deploy {
tar cf XML-Test-Suite.tar XML-Test-Suite/
- put XML-Test-Suite.tar tests/.libs/* tests/xmltest.sh $BOARD_TESTDIR/fuego.$TESTDIR/;
+ put XML-Test-Suite.tar tests/.libs/* tests/xmltest.sh $BOARD_TESTDIR/fuego.$TESTDIR/
cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; tar xf XML-Test-Suite.tar"
- cmd "mkdir -p $BOARD_TESTDIR/xmlwf";
+ cmd "mkdir -p $BOARD_TESTDIR/xmlwf"
- put xmlwf/.libs/xmlwf $BOARD_TESTDIR/xmlwf/xmlwf;
+ put xmlwf/.libs/xmlwf $BOARD_TESTDIR/xmlwf/xmlwf
}
function test_run {
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 12/19] tarball: remove unneeded tarball definitions
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (10 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 11/19] expat: remove unneeded semicolons Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 13/19] style: fix trailing spaces and indentation Daniel Sangorrin
` (7 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Benchmark.reboot/fuego_test.sh | 2 --
engine/tests/Functional.fixesproto/fuego_test.sh | 2 --
engine/tests/Functional.fuego_abort/fuego_test.sh | 2 --
engine/tests/Functional.fuego_test_phases/fuego_test.sh | 2 --
engine/tests/Functional.iputils/fuego_test.sh | 2 --
engine/tests/Functional.kmod/fuego_test.sh | 2 --
engine/tests/Functional.xorg-macros/fuego_test.sh | 2 --
7 files changed, 14 deletions(-)
diff --git a/engine/tests/Benchmark.reboot/fuego_test.sh b/engine/tests/Benchmark.reboot/fuego_test.sh
index 102c1de..e97412f 100755
--- a/engine/tests/Benchmark.reboot/fuego_test.sh
+++ b/engine/tests/Benchmark.reboot/fuego_test.sh
@@ -1,5 +1,3 @@
-tarball=none
-
function test_deploy {
put $TEST_HOME/get_reboot_time.sh $BOARD_TESTDIR/fuego.$TESTDIR/
}
diff --git a/engine/tests/Functional.fixesproto/fuego_test.sh b/engine/tests/Functional.fixesproto/fuego_test.sh
index cebd40c..ba56a16 100755
--- a/engine/tests/Functional.fixesproto/fuego_test.sh
+++ b/engine/tests/Functional.fixesproto/fuego_test.sh
@@ -1,5 +1,3 @@
-#tarball=images.tar.gz
-
function test_build {
echo "#!/bin/bash
if [ -f /usr/include/X11/extensions/xfixesproto.h ]; then echo 'TEST-1 OK'; else echo 'TEST-1 FAIL'; fi;
diff --git a/engine/tests/Functional.fuego_abort/fuego_test.sh b/engine/tests/Functional.fuego_abort/fuego_test.sh
index 1d108cd..9e587ee 100755
--- a/engine/tests/Functional.fuego_abort/fuego_test.sh
+++ b/engine/tests/Functional.fuego_abort/fuego_test.sh
@@ -1,7 +1,5 @@
#!/bin/bash
-tarball=none
-
function test_pre_check {
# if you don't use nohup, ssh transport will hang trying to background fuegosleep
is_on_target nohup PROGRAM_NOHUP /usr/bin
diff --git a/engine/tests/Functional.fuego_test_phases/fuego_test.sh b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
index 8834a3a..b09ba7a 100755
--- a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
+++ b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
@@ -1,7 +1,5 @@
#!/bin/bash
-tarball=none
-
export PHASE_STR="script load"
# use string concatenation to test phase ordering
diff --git a/engine/tests/Functional.iputils/fuego_test.sh b/engine/tests/Functional.iputils/fuego_test.sh
index 1a043b4..db35655 100755
--- a/engine/tests/Functional.iputils/fuego_test.sh
+++ b/engine/tests/Functional.iputils/fuego_test.sh
@@ -1,5 +1,3 @@
-#tarball=testtmp.tar.gz
-
function test_build {
echo "testtmp.sh build"
}
diff --git a/engine/tests/Functional.kmod/fuego_test.sh b/engine/tests/Functional.kmod/fuego_test.sh
index d9236ad..bb2e796 100755
--- a/engine/tests/Functional.kmod/fuego_test.sh
+++ b/engine/tests/Functional.kmod/fuego_test.sh
@@ -1,5 +1,3 @@
-#tarball=kmod.tar.gz
-
function test_build {
echo " test_build_kmod.sh "
}
diff --git a/engine/tests/Functional.xorg-macros/fuego_test.sh b/engine/tests/Functional.xorg-macros/fuego_test.sh
index df10fcd..b2cd824 100755
--- a/engine/tests/Functional.xorg-macros/fuego_test.sh
+++ b/engine/tests/Functional.xorg-macros/fuego_test.sh
@@ -1,5 +1,3 @@
-#tarball=images.tar.gz
-
function test_build {
echo "#!/bin/bash
if [ -f /usr/share/aclocal/xorg-macros.m4 ]; then echo 'TEST-1 OK'; else echo 'TEST-1 FAIL'; fi;
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 13/19] style: fix trailing spaces and indentation
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (11 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 12/19] tarball: remove unneeded tarball definitions Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 14/19] parser: allow parser.py to skip processing Daniel Sangorrin
` (6 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Functional.aiostress/fuego_test.sh | 8 +++----
engine/tests/Functional.ipv6connect/fuego_test.sh | 4 ++--
engine/tests/Functional.linus_stress/fuego_test.sh | 8 +++----
engine/tests/Functional.net-tools/fuego_test.sh | 26 +++++++++-------------
engine/tests/Functional.rmaptest/fuego_test.sh | 14 ++++++------
5 files changed, 28 insertions(+), 32 deletions(-)
diff --git a/engine/tests/Functional.aiostress/fuego_test.sh b/engine/tests/Functional.aiostress/fuego_test.sh
index 0e48c6b..6dbe690 100755
--- a/engine/tests/Functional.aiostress/fuego_test.sh
+++ b/engine/tests/Functional.aiostress/fuego_test.sh
@@ -1,20 +1,20 @@
tarball=aiostress.tar.gz
function test_build {
- $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio aiostress.c -o aiostress
+ $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio aiostress.c -o aiostress
}
function test_deploy {
- put aiostress $BOARD_TESTDIR/fuego.$TESTDIR/
+ put aiostress $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
assert_define FUNCTIONAL_AIOSTRESS_MOUNT_BLOCKDEV
assert_define FUNCTIONAL_AIOSTRESS_MOUNT_POINT
assert_define FUNCTIONAL_AIOSTRESS_SIZE
-
+
hd_test_mount_prepare $FUNCTIONAL_AIOSTRESS_MOUNT_BLOCKDEV $FUNCTIONAL_AIOSTRESS_MOUNT_POINT
report "cd $FUNCTIONAL_AIOSTRESS_MOUNT_POINT/fuego.$TESTDIR; $BOARD_TESTDIR/fuego.$TESTDIR/aiostress -s $FUNCTIONAL_AIOSTRESS_SIZE ./testfile"
-
+
hd_test_clean_umount $FUNCTIONAL_AIOSTRESS_MOUNT_BLOCKDEV $FUNCTIONAL_AIOSTRESS_MOUNT_POINT
}
diff --git a/engine/tests/Functional.ipv6connect/fuego_test.sh b/engine/tests/Functional.ipv6connect/fuego_test.sh
index 22454e9..95e2755 100755
--- a/engine/tests/Functional.ipv6connect/fuego_test.sh
+++ b/engine/tests/Functional.ipv6connect/fuego_test.sh
@@ -5,9 +5,9 @@ function test_build {
}
function test_deploy {
- put ipv6connect $BOARD_TESTDIR/fuego.$TESTDIR/
+ put ipv6connect $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ipv6connect"
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./ipv6connect"
}
diff --git a/engine/tests/Functional.linus_stress/fuego_test.sh b/engine/tests/Functional.linus_stress/fuego_test.sh
index 15265c4..0c68c42 100755
--- a/engine/tests/Functional.linus_stress/fuego_test.sh
+++ b/engine/tests/Functional.linus_stress/fuego_test.sh
@@ -5,14 +5,14 @@ function test_build {
}
function test_deploy {
- put linus_stress $BOARD_TESTDIR/fuego.$TESTDIR/
+ put linus_stress $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
- assert_define FUNCTIONAL_LINUS_STRESS_DIRTY_BG
- assert_define FUNCTIONAL_LINUS_STRESS_DIRTY
+ assert_define FUNCTIONAL_LINUS_STRESS_DIRTY_BG
+ assert_define FUNCTIONAL_LINUS_STRESS_DIRTY
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; \
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; \
cat /proc/sys/vm/dirty_ratio > ./dirty_ratio; \
cat /proc/sys/vm/dirty_background_ratio > ./dirty_bg; \
echo $FUNCTIONAL_LINUS_STRESS_DIRTY > /proc/sys/vm/dirty_ratio || echo 'WARNING: /proc/sys/vm/dirty_ratio is read-only'; \
diff --git a/engine/tests/Functional.net-tools/fuego_test.sh b/engine/tests/Functional.net-tools/fuego_test.sh
index 06c80b2..d73e661 100755
--- a/engine/tests/Functional.net-tools/fuego_test.sh
+++ b/engine/tests/Functional.net-tools/fuego_test.sh
@@ -1,23 +1,19 @@
#!/bin/bash
function test_run {
- report "if arp; then echo 'TEST-1 OK'; else echo 'TEST-1 FAIL'; fi; \
- if hostname; then echo 'TEST-2 OK'; else echo 'TEST-2 FAIL'; fi; \
- if ifconfig; then echo 'TEST-3 OK'; else echo 'TEST-3 FAIL'; fi; \
- if ipmaddr; then echo 'TEST-4 OK'; else echo 'TEST-4 FAIL'; fi; \
- if iptunnel; then echo 'TEST-5 OK'; else echo 'TEST-5 FAIL'; fi; \
- if netstat; then echo 'TEST-6 OK'; else echo 'TEST-6 FAIL'; fi; \
- if route; then echo 'TEST-7 OK'; else echo 'TEST-7 FAIL'; fi"
+ report "if arp; then echo 'TEST-1 OK'; else echo 'TEST-1 FAIL'; fi; \
+ if hostname; then echo 'TEST-2 OK'; else echo 'TEST-2 FAIL'; fi; \
+ if ifconfig; then echo 'TEST-3 OK'; else echo 'TEST-3 FAIL'; fi; \
+ if ipmaddr; then echo 'TEST-4 OK'; else echo 'TEST-4 FAIL'; fi; \
+ if iptunnel; then echo 'TEST-5 OK'; else echo 'TEST-5 FAIL'; fi; \
+ if netstat; then echo 'TEST-6 OK'; else echo 'TEST-6 FAIL'; fi; \
+ if route; then echo 'TEST-7 OK'; else echo 'TEST-7 FAIL'; fi"
}
function test_processing {
- P_CRIT="TEST.*OK"
- N_CRIT="TEST.*FAIL"
+ P_CRIT="TEST.*OK"
+ N_CRIT="TEST.*FAIL"
- log_compare "$TESTDIR" "7" "${P_CRIT}" "p"
- log_compare "$TESTDIR" "0" "${N_CRIT}" "n"
+ log_compare "$TESTDIR" "7" "${P_CRIT}" "p"
+ log_compare "$TESTDIR" "0" "${N_CRIT}" "n"
}
-
-
-
-
diff --git a/engine/tests/Functional.rmaptest/fuego_test.sh b/engine/tests/Functional.rmaptest/fuego_test.sh
index 7dfeb69..d502524 100755
--- a/engine/tests/Functional.rmaptest/fuego_test.sh
+++ b/engine/tests/Functional.rmaptest/fuego_test.sh
@@ -5,15 +5,15 @@ function test_build {
}
function test_deploy {
- put rmap-test $BOARD_TESTDIR/fuego.$TESTDIR/
+ put rmap-test $BOARD_TESTDIR/fuego.$TESTDIR/
}
function test_run {
- assert_define FUNCTIONAL_RMAPTEST_ITERATIONS
- assert_define FUNCTIONAL_RMAPTEST_VMA
- assert_define FUNCTIONAL_RMAPTEST_VMA_SIZE
- assert_define FUNCTIONAL_RMAPTEST_TASKS
- assert_define FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS
+ assert_define FUNCTIONAL_RMAPTEST_ITERATIONS
+ assert_define FUNCTIONAL_RMAPTEST_VMA
+ assert_define FUNCTIONAL_RMAPTEST_VMA_SIZE
+ assert_define FUNCTIONAL_RMAPTEST_TASKS
+ assert_define FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS
- report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./rmap-test -h -i$FUNCTIONAL_RMAPTEST_ITERATIONS -n$FUNCTIONAL_RMAPTEST_VMA -s$FUNCTIONAL_RMAPTEST_VMA_SIZE -t$FUNCTIONAL_RMAPTEST_TASKS -V$FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS -v file1.dat"
+ report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./rmap-test -h -i$FUNCTIONAL_RMAPTEST_ITERATIONS -n$FUNCTIONAL_RMAPTEST_VMA -s$FUNCTIONAL_RMAPTEST_VMA_SIZE -t$FUNCTIONAL_RMAPTEST_TASKS -V$FUNCTIONAL_RMAPTEST_VMAS_FOR_PROCESS -v file1.dat"
}
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 14/19] parser: allow parser.py to skip processing
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (12 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 13/19] style: fix trailing spaces and indentation Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 15/19] LTP: copy target_bin before modifying it Daniel Sangorrin
` (5 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
For some specs the parser.py may want to just skip. Returning
a 0 would cause an error on the execution of dataload.py. For
that reason I added another return value (3) to specify that
the parser.py didn't need to run.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index 188408d..1d7bce9 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -359,13 +359,12 @@ function processing {
PYTHON_ARGS="-W ignore::DeprecationWarning -W ignore::UserWarning"
if [ -e "$TEST_HOME/parser.py" ] ; then
- # FIXTHIS: make sure that json is generated even on failures
run_python $PYTHON_ARGS $FUEGO_CORE/engine/tests/${TESTDIR}/parser.py && rc=0 || rc=$?
else
run_python $PYTHON_ARGS $FUEGO_CORE/engine/scripts/generic_parser.py $RETURN_VALUE && rc=0 || rc=$?
fi
- # return codes: 0 (everything ok), 1 (problem while parsing, see log), 2 (the results didn't satisfy the threshold)
+ # return codes: 0 (everything ok), 1 (problem while parsing, see log), 2 (the results didn't satisfy the threshold), 3 (parser does not need to run)
if [ $rc -eq 0 ] || [ $rc -eq 2 ]; then
# store results as a json file fro the flot plugin
run_python $PYTHON_ARGS $FUEGO_CORE/engine/scripts/parser/dataload.py && rc=0 || echo "dataload.py didn't work properly"
@@ -377,8 +376,11 @@ function processing {
RETURN_VALUE=1
fi
else
- echo "ERROR: problem while running the parser"
- RETURN_VALUE=1
+ if [ $rc -eq 3 ]; then
+ echo "parser.py does not need to run"
+ else
+ abort_job "ERROR: problem while running the parser"
+ fi
fi
# make a convenience link to the Jenkins console log, if the log doesn't exist
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 15/19] LTP: copy target_bin before modifying it
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (13 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 14/19] parser: allow parser.py to skip processing Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 16/19] LTP: fix the buildonly and runonly cases Daniel Sangorrin
` (4 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
This is necessary for the Rebuild flag to work properly
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Functional.LTP/fuego_test.sh | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index f7c9ced..3d3d360 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -99,20 +99,22 @@ function test_deploy {
echo "Deploying LTP"
# the syscalls group occupies by far the largest space so remove it if unneeded
+ cp -r target_bin ltp
cp_syscalls=$(echo $FUNCTIONAL_LTP_TESTS | awk '{print match($0,"syscalls")}')
if [ $cp_syscalls -eq 0 ]; then
echo "Removing syscalls binaries"
- awk '/^[^#]/ { print "rm -f target_bin/testcases/bin/" $2 }' target_bin/runtest/syscalls | sh
+ awk '/^[^#]/ { print "rm -f ltp/testcases/bin/" $2 }' ltp/runtest/syscalls | sh
fi
if [ "$FUNCTIONAL_LTP_BUILDONLY" = "true" ]; then
echo "Creating LTP binary tarball"
- mv target_bin ltp
tar zcvpf ltp.tar.gz ltp/
mv ltp.tar.gz ${LOGDIR}/
else
- put target_bin/* $BOARD_TESTDIR/fuego.$TESTDIR/
+ put ltp/* $BOARD_TESTDIR/fuego.$TESTDIR/
fi
+
+ rm -rf ltp
else
echo "Skipping LTP deploy"
fi
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 16/19] LTP: fix the buildonly and runonly cases
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (14 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 15/19] LTP: copy target_bin before modifying it Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 17/19] LTP: add support for skipping certain test cases Daniel Sangorrin
` (3 subsequent siblings)
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
Previous changes had broken the buildonly/runonly support
so fix it.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Functional.LTP/fuego_test.sh | 14 +++++++-------
engine/tests/Functional.LTP/parser.py | 4 ++--
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 3d3d360..8ea78c7 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -161,10 +161,13 @@ function test_run {
}
function test_fetch_results {
- echo "Fetching LTP results"
- echo "PWD: $PWD"
- echo "Triplet: $WORKSPACE/$TRIPLET"
- get $FUNCTIONAL_LTP_RUNFOLDER/result .
+ if [ "$FUNCTIONAL_LTP_BUILDONLY" = "false" ]; then
+ echo "Fetching LTP results"
+ rm -rf result/
+ get $FUNCTIONAL_LTP_RUNFOLDER/result .
+ else
+ echo "Skip fetching LTP results"
+ fi
}
function test_processing {
@@ -178,11 +181,8 @@ function test_processing {
[ -e results.xlsx ] && cp results.xlsx ${LOGDIR}/results.xlsx
[ -e rt.log ] && cp rt.log $FUEGO_RW/logs/$JOB_NAME/
cd ..
- rm -rf result/
else
echo "Processing LTP build log"
log_compare "$TESTDIR" "1836" "compile PASSED$" "p"
fi
}
-
-
diff --git a/engine/tests/Functional.LTP/parser.py b/engine/tests/Functional.LTP/parser.py
index 6f0638c..feab36a 100755
--- a/engine/tests/Functional.LTP/parser.py
+++ b/engine/tests/Functional.LTP/parser.py
@@ -77,8 +77,8 @@ def read_output (test_category, test_case):
try:
os.chdir('./result')
except:
- print "ERROR: no result directory"
- sys.exit(1)
+ print "WARNING: no result directory (probably a build only test)."
+ sys.exit(3)
tests = os.listdir('.')
tests.sort()
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 17/19] LTP: add support for skipping certain test cases
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (15 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 16/19] LTP: fix the buildonly and runonly cases Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 22:57 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 18/19] LTP: add a spec for docker Daniel Sangorrin
` (2 subsequent siblings)
19 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
The implementation reuses the -S option available already
in LTP's script.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Functional.LTP/fuego_test.sh | 8 +++++++-
engine/tests/Functional.LTP/ltp_target_run.sh | 1 +
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 8ea78c7..35dec5a 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -106,6 +106,13 @@ function test_deploy {
awk '/^[^#]/ { print "rm -f ltp/testcases/bin/" $2 }' ltp/runtest/syscalls | sh
fi
+ echo "# skip this test cases" > ltp/skiplist.txt
+ if [ ! -z ${FUNCTIONAL_LTP_SKIPLIST+x} ]; then
+ for item in $FUNCTIONAL_LTP_SKIPLIST; do
+ echo $item >> ltp/skiplist.txt
+ done
+ fi
+
if [ "$FUNCTIONAL_LTP_BUILDONLY" = "true" ]; then
echo "Creating LTP binary tarball"
tar zcvpf ltp.tar.gz ltp/
@@ -152,7 +159,6 @@ function test_run {
FUNCTIONAL_LTP_RUNFOLDER="$BOARD_TESTDIR/fuego.$TESTDIR"
fi
- # FIXTHIS: add blacklist of tests (kill10 etc)
# Let some of the tests fail, the information will be in the result xlsx file
report "cd $FUNCTIONAL_LTP_RUNFOLDER; TESTS=\"$TESTS\"; PTSTESTS=\"$PTSTESTS\"; RTTESTS=\"$RTTESTS\"; . ./ltp_target_run.sh"
else
diff --git a/engine/tests/Functional.LTP/ltp_target_run.sh b/engine/tests/Functional.LTP/ltp_target_run.sh
index 6a67d35..8907ebe 100755
--- a/engine/tests/Functional.LTP/ltp_target_run.sh
+++ b/engine/tests/Functional.LTP/ltp_target_run.sh
@@ -23,6 +23,7 @@ for i in ${TESTS}; do
-l ${OUTPUT_DIR}/${i}/result.log \
-o ${OUTPUT_DIR}/${i}/output.log \
-d ${TMP_DIR} \
+ -S ./skiplist.txt \
-f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1
rm -rf ${TMP_DIR}/*
done
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 18/19] LTP: add a spec for docker
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (16 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 17/19] LTP: add support for skipping certain test cases Daniel Sangorrin
@ 2017-05-12 8:19 ` Daniel Sangorrin
2017-05-12 8:20 ` [Fuego] [PATCH 19/19] LTP: add a fixthis for the -t flag Daniel Sangorrin
2017-05-12 22:52 ` [Fuego] New batch of patches Bird, Timothy
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:19 UTC (permalink / raw)
To: fuego
The main reason to add this spec is to skip a couple
of tests (kill10 and keyctl01) when running LTP on docker.
There are also many tests that cannot be run because they require
root permissions but we want to see failing and TCONF results
on the output for verifying that everything is working fine.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/overlays/testplans/testplan_docker.json | 1 +
engine/tests/Functional.LTP/spec.json | 5 +++++
2 files changed, 6 insertions(+)
diff --git a/engine/overlays/testplans/testplan_docker.json b/engine/overlays/testplans/testplan_docker.json
index 488417d..b90a655 100644
--- a/engine/overlays/testplans/testplan_docker.json
+++ b/engine/overlays/testplans/testplan_docker.json
@@ -53,6 +53,7 @@
},
{
"testName": "Functional.LTP",
+ "spec": "docker",
"timeout": "100m"
},
{
diff --git a/engine/tests/Functional.LTP/spec.json b/engine/tests/Functional.LTP/spec.json
index 47668c2..7e02158 100644
--- a/engine/tests/Functional.LTP/spec.json
+++ b/engine/tests/Functional.LTP/spec.json
@@ -5,6 +5,11 @@
"tests": "syscalls SEM",
"extra_success_links": {"xlsx": "results.xlsx"}
},
+ "docker": {
+ "tests": "syscalls",
+ "skiplist": "kill10 keyctl01",
+ "extra_success_links": {"xlsx": "results.xlsx"}
+ },
"selection": {
"tests": "syscalls fs pipes sched timers dio mm ipc pty AIO MSG SEM SIG THR TMR TPS",
"extra_success_links": {"xlsx": "results.xlsx"}
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [Fuego] [PATCH 19/19] LTP: add a fixthis for the -t flag
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (17 preceding siblings ...)
2017-05-12 8:19 ` [Fuego] [PATCH 18/19] LTP: add a spec for docker Daniel Sangorrin
@ 2017-05-12 8:20 ` Daniel Sangorrin
2017-05-12 22:52 ` [Fuego] New batch of patches Bird, Timothy
19 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-12 8:20 UTC (permalink / raw)
To: fuego
LTP has a flag where we can limit the duration of each
execution. This can be useful to specify per-group timeouts.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Functional.LTP/ltp_target_run.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/engine/tests/Functional.LTP/ltp_target_run.sh b/engine/tests/Functional.LTP/ltp_target_run.sh
index 8907ebe..d8e3852 100755
--- a/engine/tests/Functional.LTP/ltp_target_run.sh
+++ b/engine/tests/Functional.LTP/ltp_target_run.sh
@@ -16,6 +16,7 @@ echo y | ./IDcheck.sh
echo "ltp_target_run: ${TESTS} | ${PTSTESTS} | ${RTTESTS}"
+# FIXTHIS: add -t option for limiting the duration of each test group execution
for i in ${TESTS}; do
echo "ltp_target_run: doing test $i"
mkdir -p ${OUTPUT_DIR}/${i}
--
2.7.4
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [Fuego] New batch of patches
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
` (18 preceding siblings ...)
2017-05-12 8:20 ` [Fuego] [PATCH 19/19] LTP: add a fixthis for the -t flag Daniel Sangorrin
@ 2017-05-12 22:52 ` Bird, Timothy
19 siblings, 0 replies; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 22:52 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Friday, May 12, 2017 1:20 AM
>
> Please consider this patches for fuego-core. Some of them are
> a refinement over the previous patches that I sent in a rush, and
> some of them are new. I have kept the concurrent_check function
> for now.
Thanks for these. I accepted all of them, but have a few questions
on one or two of them. I post those questions on the individual patches.
> [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for
> [PATCH 02/19] functions:fail_check_cases: add message when no fail
> This one originally included a fix like this:
> # expr returns 1 if the expression is 0, which may be incompatible with set
> -e
> - fcc=`expr $fcc - 1` || true
> But I removed it because it caused a merge conflict with your new
> patch. I think the problem doesn't arise with your patch.
>
> [PATCH 03/19] ft2demos: fix test and add to docker
> [PATCH 04/19] unpack: remove nostrip option
> [PATCH 05/19] unpack: move code related to spec-defined tarballs to
> [PATCH 06/19] bc: test that specs can override the tarball variable
> [PATCH 07/19] unpack: handle possible errors
> [PATCH 08/19] main: remove ReBuild check
> [PATCH 09/19] rebuild: make it simpler for test developers
> This is the biggest change in this series, but I think it solves one
> of those problems that you mentioned when you see the same pattern
> repeated all over that should be shared somehow. You were right
> on your concerns about the LTP test not being able to handle the
> flag, but I fixed that PATCH 15/19.
>
> [PATCH 10/19] call_if_present: tests do not need to implement dummy
> You mentioned that you liked to have all functions as in a
> template, but I think this is also one of those patterns and
> we should remove them. Instead we should provide a nice template
> in the documentation.
>
> [PATCH 11/19] expat: remove unneeded semicolons
> This test is not working on docker for some reason. The OpenSSH
> test doesn't work either on docker (but I think it does on my ARM board).
> We might need to double-check or remove them from testplan_docker.
>
> [PATCH 12/19] tarball: remove unneeded tarball definitions
> [PATCH 13/19] style: fix trailing spaces and indentation
> [PATCH 14/19] parser: allow parser.py to skip processing
> This is not very beautiful but it's required (see PATCH 16/19).
>
> [PATCH 15/19] LTP: copy target_bin before modifying it
> [PATCH 16/19] LTP: fix the buildonly and runonly cases
> [PATCH 17/19] LTP: add support for skipping certain test cases
> This implements the skiplist (previously known as blacklist but
> I call it skiplist like inside the LTP project).
>
> [PATCH 18/19] LTP: add a spec for docker
> [PATCH 19/19] LTP: add a fixthis for the -t flag
> This could be useful for handling runaway processes that hang
> the CPU for ours.
Yes - I have one of these in LTP for docker, that isn't blacklisted
yet. I'll probably just blacklist the test, but I do think it shows
a real docker bug. I'd rather have the test show up as a failure
due to timeout, than just put it on a list and possibly have it
fall through the cracks.
-- Tim
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency
2017-05-12 8:19 ` [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency Daniel Sangorrin
@ 2017-05-12 22:56 ` Bird, Timothy
0 siblings, 0 replies; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 22:56 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, May 12, 2017 1:20 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to
> FAIL_REGEXP for coherency
>
> I also improved the documentation of the function fail_check_cases
> to mention that syslog (actually the diff) may also be checked
> by defining the variable use_syslog on the spec.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 7 +++++--
> engine/scripts/ovgen.py | 2 +-
> 2 files changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index 1432b5e..f9681b3 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -393,7 +393,10 @@ function processing {
> return $RETURN_VALUE
> }
>
> -# search in test log for {!JOB_NAME}_FAIL_PATTERN_n fail cases and abort
> with message {!JOB_NAME}_FAIL_MESSAGE_n if found
> +# search in testlog.txt and syslog (if use_syslog is defined in the spec)
> +# for {!TESTDIR}_FAIL_REGEXP_n fail cases (fail_regexp in the spec) and
> +# abort with message {!TESTDIR}_FAIL_MESSAGE_n (fail_message in the
> spec)
> +# if found.
> function fail_check_cases () {
> testlog="${LOGDIR}/testlog.txt"
> slog_prefix="${LOGDIR}/syslog"
> @@ -413,7 +416,7 @@ function fail_check_cases () {
>
> for n in `seq 0 $fcc`
> do
> - fpvarname="${upName}"_FAIL_PATTERN_"${n}"
> + fpvarname="${upName}"_FAIL_REGEXP_"${n}"
Who generates these names (or do they come directly from the spec files)?
I didn't see any spec file changes with this patch, and I can't find any
instances of FAIL_PATTERN in any existing files.
Is this feature just unused by current tests?
...
Ok - nevermind - these come from "fail_regexp" in the spec.
Highly agreed that the name here (and in console logs) should
be consistent to avoid confusion. Thanks!
> fpvarmsg="${upName}"_FAIL_MESSAGE_"${n}"
> fpvarslog="${upName}"_FAIL_"${n}"_SYSLOG
>
> diff --git a/engine/scripts/ovgen.py b/engine/scripts/ovgen.py
> index e514d7e..ed67667 100755
> --- a/engine/scripts/ovgen.py
> +++ b/engine/scripts/ovgen.py
> @@ -422,7 +422,7 @@ def generateSpec(ts, fout):
> fout.write(outNum + "\n")
>
> for fmsg, num in zip(ts.fail_case, range(fc_num)):
> - outPattern = "%s_FAIL_PATTERN_%s=\"%s\"" % (tNameUp, num,
> fmsg["fail_regexp"])
> + outPattern = "%s_FAIL_REGEXP_%s=\"%s\"" % (tNameUp, num,
> fmsg["fail_regexp"])
> outMessage = "%s_FAIL_MESSAGE_%s=\"%s\"" % (tNameUp, num,
> fmsg["fail_message"])
>
> if "use_syslog" in fmsg:
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 17/19] LTP: add support for skipping certain test cases
2017-05-12 8:19 ` [Fuego] [PATCH 17/19] LTP: add support for skipping certain test cases Daniel Sangorrin
@ 2017-05-12 22:57 ` Bird, Timothy
0 siblings, 0 replies; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 22:57 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
Nice. Thanks!
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, May 12, 2017 1:20 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 17/19] LTP: add support for skipping certain test
> cases
>
> The implementation reuses the -S option available already
> in LTP's script.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/tests/Functional.LTP/fuego_test.sh | 8 +++++++-
> engine/tests/Functional.LTP/ltp_target_run.sh | 1 +
> 2 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/engine/tests/Functional.LTP/fuego_test.sh
> b/engine/tests/Functional.LTP/fuego_test.sh
> index 8ea78c7..35dec5a 100755
> --- a/engine/tests/Functional.LTP/fuego_test.sh
> +++ b/engine/tests/Functional.LTP/fuego_test.sh
> @@ -106,6 +106,13 @@ function test_deploy {
> awk '/^[^#]/ { print "rm -f ltp/testcases/bin/" $2 }' ltp/runtest/syscalls
> | sh
> fi
>
> + echo "# skip this test cases" > ltp/skiplist.txt
> + if [ ! -z ${FUNCTIONAL_LTP_SKIPLIST+x} ]; then
> + for item in $FUNCTIONAL_LTP_SKIPLIST; do
> + echo $item >> ltp/skiplist.txt
> + done
> + fi
> +
> if [ "$FUNCTIONAL_LTP_BUILDONLY" = "true" ]; then
> echo "Creating LTP binary tarball"
> tar zcvpf ltp.tar.gz ltp/
> @@ -152,7 +159,6 @@ function test_run {
> FUNCTIONAL_LTP_RUNFOLDER="$BOARD_TESTDIR/fuego.$TESTDIR"
> fi
>
> - # FIXTHIS: add blacklist of tests (kill10 etc)
> # Let some of the tests fail, the information will be in the result xlsx file
> report "cd $FUNCTIONAL_LTP_RUNFOLDER; TESTS=\"$TESTS\";
> PTSTESTS=\"$PTSTESTS\"; RTTESTS=\"$RTTESTS\"; . ./ltp_target_run.sh"
> else
> diff --git a/engine/tests/Functional.LTP/ltp_target_run.sh
> b/engine/tests/Functional.LTP/ltp_target_run.sh
> index 6a67d35..8907ebe 100755
> --- a/engine/tests/Functional.LTP/ltp_target_run.sh
> +++ b/engine/tests/Functional.LTP/ltp_target_run.sh
> @@ -23,6 +23,7 @@ for i in ${TESTS}; do
> -l ${OUTPUT_DIR}/${i}/result.log \
> -o ${OUTPUT_DIR}/${i}/output.log \
> -d ${TMP_DIR} \
> + -S ./skiplist.txt \
> -f $i > ${OUTPUT_DIR}/${i}/head.log 2>&1
> rm -rf ${TMP_DIR}/*
> done
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 04/19] unpack: remove nostrip option
2017-05-12 8:19 ` [Fuego] [PATCH 04/19] unpack: remove nostrip option Daniel Sangorrin
@ 2017-05-12 22:57 ` Bird, Timothy
0 siblings, 0 replies; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 22:57 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
Good simplification!
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, May 12, 2017 1:20 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 04/19] unpack: remove nostrip option
>
> unpack used to have a "nostrip" option so that the tarball
> would be extracted to its own folder. This was used in the past
> by ft2demos which was calling build directly.
>
> The problem is that tests are not supposed to call build directly
> and therefore we would need to add yet another environment variable
> for main.sh to pass it to build. Since most tests are fine with the
> current default options, I think that tests like ft2demos should
> unpack their own tarballs during test_build instead.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 14 +++++---------
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index ac19e8e..d9e0b18 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -37,11 +37,7 @@ trap signal_handler SIGTERM SIGHUP SIGALRM SIGINT
> EXIT
> set -o errtrace
>
> # Unpacks $tarball_path/$tarball into current directory.
> -# $1 - optional flag; if set to "nostrip",
> -# the leading path components won't be stripped
> function unpack {
> - [ "$1" = "nostrip" ] && strip_opt= || strip_opt="--strip-components=1"
> -
> if [ ! -z ${tarball+x} ]; then
> case ${tarball/*./} in
> gz|tgz) key=z ;;
> @@ -49,14 +45,15 @@ function unpack {
> tar) key= ;;
> *) echo "Unknown $tarball file format. Not unpacking."; return;;
> esac
> - tar ${key}xf $TEST_HOME/$tarball $strip_opt
> + tar ${key}xf $TEST_HOME/$tarball --strip-components=1
> fi
>
> if [ ! -z ${gitrepo+x} ]; then
> if [ -z ${gitbranch+x} ]; then
> gitbranch="master"
> fi
> - # FIXTHIS: support commit ids (log and checkout)
> + # FIXTHIS: support commit ids (log and checkout)
> + # FIXTHIS: strip components like tarballs
> git clone --depth=1 --branch=$gitbranch $gitrepo
> fi
> }
> @@ -171,7 +168,6 @@ function build_error () {
> # process Rebuild flag, and unpack test sources if necessary.
> # Returns 0 if actual build needs to be performed; 1 - otherwise.
> # Build scripts must call this function in the beginning.
> -# $1 is passed directly to unpack().
> function pre_build {
> cd ${WORKSPACE}
> upName=`echo "${JOB_NAME^^}"| tr '.' '_'`
> @@ -193,7 +189,7 @@ function pre_build {
> fi
>
> if [ ! -e test_suite_ready ]; then
> - unpack $1
> + unpack
> return 0
> fi
> fi
> @@ -201,7 +197,7 @@ function pre_build {
> }
>
> function build {
> - pre_build $1
> + pre_build
> build_start_time=$(date +"%s.%N")
> test_build || return 1
> build_end_time=$(date +"%s.%N")
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
2017-05-12 8:19 ` [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers Daniel Sangorrin
@ 2017-05-12 23:02 ` Bird, Timothy
2017-05-16 2:35 ` Daniel Sangorrin
0 siblings, 1 reply; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 23:02 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
This one worries me a bit.
I like the simplification, but I'm not sure I trust the logic (see below for a question).
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, May 12, 2017 1:20 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
>
> This commit reimplements the way the "rebuild" flag is handled.
> First, it unifies the variables 'build_failed' and 'test_suite_ready'
> into a single 'fuego_test_succesfully_built' variable.
> And second, the core code inside functions.sh is in charge of
> setting this variable by reading the exit code from test_build.
>
> Therefore, the main change is that now test developers only need
> to care about the exit code returned by test_build. The variable
> fuego_test_succesfully_built is handled by the core code, so
> no need of redundant code on each test.
>
> TODO: change the documentation for test creators.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 57 +++++++++++-----------
> engine/tests/Benchmark.Dhrystone/fuego_test.sh | 6 +--
> engine/tests/Benchmark.GLMark/fuego_test.sh | 2 +-
> engine/tests/Benchmark.IOzone/fuego_test.sh | 4 +-
> engine/tests/Benchmark.Interbench/fuego_test.sh | 2 +-
> engine/tests/Benchmark.Java/fuego_test.sh | 4 --
> engine/tests/Benchmark.Stream/fuego_test.sh | 2 +-
> engine/tests/Benchmark.Whetstone/fuego_test.sh | 2 +-
> engine/tests/Benchmark.aim7/fuego_test.sh | 2 +-
> engine/tests/Benchmark.blobsallad/fuego_test.sh | 2 +-
> engine/tests/Benchmark.bonnie/fuego_test.sh | 2 +-
> engine/tests/Benchmark.cyclictest/fuego_test.sh | 2 +-
> engine/tests/Benchmark.dbench/fuego_test.sh | 2 +-
> engine/tests/Benchmark.ebizzy/fuego_test.sh | 2 +-
> engine/tests/Benchmark.ffsb/fuego_test.sh | 2 +-
> engine/tests/Benchmark.fio/fuego_test.sh | 1 -
> .../Benchmark.fuego_check_plots/fuego_test.sh | 4 --
> engine/tests/Benchmark.gtkperf/fuego_test.sh | 2 +-
> engine/tests/Benchmark.hackbench/fuego_test.sh | 2 +-
> engine/tests/Benchmark.himeno/fuego_test.sh | 2 +-
> engine/tests/Benchmark.iperf/fuego_test.sh | 2 +-
> engine/tests/Benchmark.linpack/fuego_test.sh | 2 +-
> engine/tests/Benchmark.lmbench2/fuego_test.sh | 2 +-
> engine/tests/Benchmark.nbench-byte/fuego_test.sh | 2 +-
> engine/tests/Benchmark.nbench_byte/fuego_test.sh | 2 +-
> engine/tests/Benchmark.netpipe/fuego_test.sh | 2 +-
> engine/tests/Benchmark.signaltest/fuego_test.sh | 2 +-
> engine/tests/Benchmark.tiobench/fuego_test.sh | 2 +-
> engine/tests/Benchmark.x11perf/fuego_test.sh | 2 +-
> engine/tests/Functional.LTP/fuego_test.sh | 2 -
> engine/tests/Functional.aiostress/fuego_test.sh | 2 +-
> engine/tests/Functional.bsdiff/fuego_test.sh | 4 --
> engine/tests/Functional.bzip2/fuego_test.sh | 1 -
> engine/tests/Functional.cmt/fuego_test.sh | 4 --
> engine/tests/Functional.crashme/fuego_test.sh | 2 +-
> engine/tests/Functional.curl/fuego_test.sh | 4 --
> engine/tests/Functional.expat/fuego_test.sh | 4 +-
> engine/tests/Functional.fixesproto/fuego_test.sh | 2 -
> engine/tests/Functional.fuego_abort/fuego_test.sh | 4 --
> .../Functional.fuego_board_check/fuego_test.sh | 4 --
> .../Functional.fuego_test_phases/fuego_test.sh | 1 -
> engine/tests/Functional.glib/fuego_test.sh | 2 +-
> engine/tests/Functional.glibc/fuego_test.sh | 1 -
> engine/tests/Functional.hello_world/fuego_test.sh | 2 +-
> engine/tests/Functional.imagemagick/fuego_test.sh | 1 -
> engine/tests/Functional.ipv6connect/fuego_test.sh | 2 +-
> engine/tests/Functional.jpeg/fuego_test.sh | 1 -
> engine/tests/Functional.libtar/fuego_test.sh | 1 -
> engine/tests/Functional.linus_stress/fuego_test.sh | 2 +-
> engine/tests/Functional.lwip/fuego_test.sh | 1 -
> engine/tests/Functional.pi_tests/fuego_test.sh | 2 +-
> engine/tests/Functional.pixman/fuego_test.sh | 1 -
> engine/tests/Functional.pppd/fuego_test.sh | 1 -
> engine/tests/Functional.protobuf/fuego_test.sh | 1 -
> engine/tests/Functional.rmaptest/fuego_test.sh | 2 +-
> engine/tests/Functional.scifab/fuego_test.sh | 4 --
> engine/tests/Functional.scrashme/fuego_test.sh | 10 ++--
> engine/tests/Functional.sdhi_0/fuego_test.sh | 4 --
> engine/tests/Functional.stress/fuego_test.sh | 2 +-
> engine/tests/Functional.synctest/fuego_test.sh | 2 +-
> engine/tests/Functional.thrift/fuego_test.sh | 1 -
> engine/tests/Functional.xorg-macros/fuego_test.sh | 1 -
> engine/tests/Functional.zlib/fuego_test.sh | 2 +-
> engine/tests/OpenSSL/openssl.sh | 1 -
> engine/tests/netperf/netperf.sh | 2 +-
> 65 files changed, 75 insertions(+), 130 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index d5bf969..7ff77bc 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -159,12 +159,13 @@ function dump_syslogs {
> ov_rootfs_logread "$@"
> }
>
> +# Wait if an instance of the same job (board-test-spec) is running
> function concurrent_check {
> LOCKFILE="$WORKSPACE/$TRIPLET.build.lock"
>
> if [ -e ${LOCKFILE} ]; then
>
> - while $(wget -qO- "$(cat
> ${LOCKFILE})/api/xml?xpath=*/building/text%28%29") && [ ! -e
> test_suite_ready ]
> + while $(wget -qO- "$(cat
> ${LOCKFILE})/api/xml?xpath=*/building/text%28%29") && [ ! -e
> fuego_test_succesfully_built ]
> do
> sleep 5
> done
> @@ -173,43 +174,41 @@ function concurrent_check {
> echo "${BUILD_URL}" > ${LOCKFILE}
> }
>
> -function build_error () {
> - touch build_failed
> - abort_job "Build failed: $@"
> -}
> -
> -# Wait for other builds of the same test running in parallel,
> # process Rebuild flag, and unpack test sources if necessary.
> -# Returns 0 if actual build needs to be performed; 1 - otherwise.
> -# Build scripts must call this function in the beginning.
> function pre_build {
> - cd ${WORKSPACE}
> + cd ${WORKSPACE}
>
> - mkdir -p $TRIPLET && cd $TRIPLET
> - if concurrent_check; then
> - [ "$Rebuild" = "true" ] && rm -rf *
> + mkdir -p $TRIPLET && cd $TRIPLET
>
> - if [ -e build_failed ]; then
> - rm $TRIPLET/build_failed
> - rm -rf *
> - fi
> + concurrent_check
>
> - if [ ! -e test_suite_ready ]; then
> - unpack || abort_job "Error while unpacking"
> - return 0
> + if [ "$Rebuild" = "false" ] && [ -e fuego_test_succesfully_built ]; then
> + echo "The test is already built"
> + else
> + rm -rf *
> + unpack || abort_job "Error while unpacking"
> fi
> - fi
> - return 1
> }
>
> function build {
> - pre_build
> - build_start_time=$(date +"%s.%N")
> - test_build || return 1
> - build_end_time=$(date +"%s.%N")
> - build_duration=$(python -c "print $build_end_time - $build_start_time")
> - echo "Fuego test_build duration=$build_duration seconds"
> - post_build
> + local ret
> +
> + pre_build
> + if [ -e fuego_test_succesfully_built ]; then
> + build_duration=0
> + else
> + build_start_time=$(date +"%s.%N")
> + call_if_present test_build && ret=0 || ret=$?
> + build_end_time=$(date +"%s.%N")
> + build_duration=$(python -c "print $build_end_time -
> $build_start_time")
> + if [ $ret -eq 0 ]; then
> + touch fuego_test_succesfully_built
> + else
> + abort_job "ERROR: test_build returned $ret"
> + fi
> + fi
> + echo "Fuego test_build duration=$build_duration seconds"
> + post_build
> }
>
> function post_build {
> diff --git a/engine/tests/Benchmark.Dhrystone/fuego_test.sh
> b/engine/tests/Benchmark.Dhrystone/fuego_test.sh
> index bc3f4c8..5f5f2ed 100755
> --- a/engine/tests/Benchmark.Dhrystone/fuego_test.sh
> +++ b/engine/tests/Benchmark.Dhrystone/fuego_test.sh
> @@ -5,14 +5,14 @@ function test_pre_check {
> }
>
> function test_build {
> - patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return 1
> + patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return
Does this end up returning the return value from patch?
How does this interact with 'set -e'? I think this should work
since bash doesn't trap command errors when the command
is in a logic expression. But does an error trap from the
non-zero return from test_build?
> CFLAGS+=" -DTIME"
> LDFLAGS+=" -lm"
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch
> test_suite_ready || return 1
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
> }
>
> function test_deploy {
> - put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/ || return 1
> + put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/
> }
>
> function test_run {
> diff --git a/engine/tests/Benchmark.GLMark/fuego_test.sh
> b/engine/tests/Benchmark.GLMark/fuego_test.sh
> index f409109..7deb6e2 100755
> --- a/engine/tests/Benchmark.GLMark/fuego_test.sh
> +++ b/engine/tests/Benchmark.GLMark/fuego_test.sh
> @@ -7,7 +7,7 @@ function test_build {
> -L${SDKROOT}/usr/lib -Wl,-rpath-link=${SDKROOT}/usr/lib \
> -L${SDKROOT}/lib \
> *.cpp -o glmark -lSDL -lGL \
> - -lGLU -lGLEW && touch test_suite_ready || build_error "error while
> building test"
> + -lGLU -lGLEW
> # -Wl,--allow-shlib-undefined *.cpp -o glmark -lSDL -lGL \
> }
>
> diff --git a/engine/tests/Benchmark.IOzone/fuego_test.sh
> b/engine/tests/Benchmark.IOzone/fuego_test.sh
> index fe06e75..16b5c61 100755
> --- a/engine/tests/Benchmark.IOzone/fuego_test.sh
> +++ b/engine/tests/Benchmark.IOzone/fuego_test.sh
> @@ -11,10 +11,10 @@ function test_build {
> TARGET=linux-AMD64
> else
> echo "platform based on $ARCHITECTURE is not supported by
> benchmark"
> - build_error "error while building test"
> + return 1
> fi
>
> - make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready ||
> build_error "error while building test"
> + make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.Interbench/fuego_test.sh
> b/engine/tests/Benchmark.Interbench/fuego_test.sh
> index 5f950e2..31e6d2f 100755
> --- a/engine/tests/Benchmark.Interbench/fuego_test.sh
> +++ b/engine/tests/Benchmark.Interbench/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=interbench-0.31.tar.bz2
>
> function test_build {
> patch -p0 < $TEST_HOME/interbench.c.patch
> - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while
> building test"
> + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.Java/fuego_test.sh
> b/engine/tests/Benchmark.Java/fuego_test.sh
> index 7978263..22cdbb7 100755
> --- a/engine/tests/Benchmark.Java/fuego_test.sh
> +++ b/engine/tests/Benchmark.Java/fuego_test.sh
> @@ -1,9 +1,5 @@
> tarball=java_perf.tar
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> put *.jar $BOARD_TESTDIR/fuego.$TESTDIR/
> }
> diff --git a/engine/tests/Benchmark.Stream/fuego_test.sh
> b/engine/tests/Benchmark.Stream/fuego_test.sh
> index ddeb26c..9f4c04c 100755
> --- a/engine/tests/Benchmark.Stream/fuego_test.sh
> +++ b/engine/tests/Benchmark.Stream/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=stream.tar.bz2
>
> function test_build {
> - make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR"
> RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch
> test_suite_ready || build_error "error while building test"
> + make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR"
> RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.Whetstone/fuego_test.sh
> b/engine/tests/Benchmark.Whetstone/fuego_test.sh
> index 925bfcb..5e791f0 100755
> --- a/engine/tests/Benchmark.Whetstone/fuego_test.sh
> +++ b/engine/tests/Benchmark.Whetstone/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=Whetstone.tar.bz2
>
> function test_build {
> CFLAGS+=" -DTIME"
> - make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> LIBS=" -lm" && touch test_suite_ready || build_error "error while building
> test"
> + make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> LIBS=" -lm"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.aim7/fuego_test.sh
> b/engine/tests/Benchmark.aim7/fuego_test.sh
> index 2cee552..87f9df9 100755
> --- a/engine/tests/Benchmark.aim7/fuego_test.sh
> +++ b/engine/tests/Benchmark.aim7/fuego_test.sh
> @@ -3,7 +3,7 @@ tarball=osdl-aim-7.0.1.13.tar.gz
> function test_build {
> ./bootstrap
> PKG_CONFIG_PATH=${SDKROOT}/usr/lib/pkgconfig
> PKG_CONFIG_ALLOW_SYSTEM_LIBS=1
> PKG_CONFIG_SYSROOT_DIR=${SDKROOT} ./configure --host=$HOST --
> build=`uname -m`-linux-gnu LDFLAGS=-L${SDKROOT}/usr/lib CPPFLAGS=-
> I${SDKROOT}/usr/include CFLAGS=-I${SDKROOT}/usr/include LIBS=-laio --
> prefix=$BOARD_TESTDIR/$TESTDIR --
> datarootdir=$BOARD_TESTDIR/$TESTDIR
> - make && touch test_suite_ready || build_error "error while building
> test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.blobsallad/fuego_test.sh
> b/engine/tests/Benchmark.blobsallad/fuego_test.sh
> index ed5273d..f849946 100755
> --- a/engine/tests/Benchmark.blobsallad/fuego_test.sh
> +++ b/engine/tests/Benchmark.blobsallad/fuego_test.sh
> @@ -5,7 +5,7 @@ function test_build {
> patch -p0 -N -s < $TEST_HOME/blobsallad.auto.patch
> patch -p0 -N -s < $TEST_HOME/bs_main.c.patch
>
> - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" SDKROOT="$SDKROOT" && touch test_suite_ready ||
> build_error "error while building test"
> + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" SDKROOT="$SDKROOT"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.bonnie/fuego_test.sh
> b/engine/tests/Benchmark.bonnie/fuego_test.sh
> index 562088a..d94dce4 100755
> --- a/engine/tests/Benchmark.bonnie/fuego_test.sh
> +++ b/engine/tests/Benchmark.bonnie/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=bonnie++-1.03e.tar.gz
>
> function test_build {
> ./configure --host=$HOST --build=`uname -m`-linux-gnu;
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.cyclictest/fuego_test.sh
> b/engine/tests/Benchmark.cyclictest/fuego_test.sh
> index 657de9b..2631a82 100755
> --- a/engine/tests/Benchmark.cyclictest/fuego_test.sh
> +++ b/engine/tests/Benchmark.cyclictest/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=cyclictest.tar.gz
>
> function test_build {
> - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch
> test_suite_ready || build_error "error while building test"
> + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.dbench/fuego_test.sh
> b/engine/tests/Benchmark.dbench/fuego_test.sh
> index d84c820..f7160f9 100755
> --- a/engine/tests/Benchmark.dbench/fuego_test.sh
> +++ b/engine/tests/Benchmark.dbench/fuego_test.sh
> @@ -3,7 +3,7 @@ tarball=dbench-3.04.tar.gz
> function test_build {
> patch -N -s -p1 < $TEST_HOME/dbench_startup.patch
> ./configure --host=$HOST --build=`uname -m`-linux-gnu
> CFLAGS="$CFLAGS";
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.ebizzy/fuego_test.sh
> b/engine/tests/Benchmark.ebizzy/fuego_test.sh
> index 1e1e7ac..28b9fc0 100755
> --- a/engine/tests/Benchmark.ebizzy/fuego_test.sh
> +++ b/engine/tests/Benchmark.ebizzy/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=ebizzy-0.3.tar.gz
>
> function test_build {
> - $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c && touch
> test_suite_ready || build_error "error while building test"
> + $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.ffsb/fuego_test.sh
> b/engine/tests/Benchmark.ffsb/fuego_test.sh
> index 2037c8f..80a157d 100755
> --- a/engine/tests/Benchmark.ffsb/fuego_test.sh
> +++ b/engine/tests/Benchmark.ffsb/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=ffsb-6.0-rc2.tar.bz2
>
> function test_build {
> ./configure --host=$HOST --build=`uname -m`-linux-gnu CC=$CC AR=$AR
> RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP
> CFLAGS="$CFLAGS";
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.fio/fuego_test.sh
> b/engine/tests/Benchmark.fio/fuego_test.sh
> index 344e445..039ec15 100755
> --- a/engine/tests/Benchmark.fio/fuego_test.sh
> +++ b/engine/tests/Benchmark.fio/fuego_test.sh
> @@ -14,7 +14,6 @@ function test_build {
> # >/dev/null
>
> make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP"
> - touch ../test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> index 562b659..80e4999 100644
> --- a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> +++ b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> @@ -1,7 +1,3 @@
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> return 0
> }
> diff --git a/engine/tests/Benchmark.gtkperf/fuego_test.sh
> b/engine/tests/Benchmark.gtkperf/fuego_test.sh
> index 07a4487..45b07bb 100755
> --- a/engine/tests/Benchmark.gtkperf/fuego_test.sh
> +++ b/engine/tests/Benchmark.gtkperf/fuego_test.sh
> @@ -9,7 +9,7 @@ function test_build {
> cd ..
> export PATH=/usr/local/bin:$PATH
> ./configure --prefix=$BOARD_TESTDIR/$TESTDIR --host=$HOST --
> build=`uname -m`-linux-gnu --target=$HOST
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.hackbench/fuego_test.sh
> b/engine/tests/Benchmark.hackbench/fuego_test.sh
> index 3b45dd3..385f533 100755
> --- a/engine/tests/Benchmark.hackbench/fuego_test.sh
> +++ b/engine/tests/Benchmark.hackbench/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=hackbench.tar.gz
>
> function test_build {
> - $CC -lpthread hackbench.c -o hackbench && touch test_suite_ready ||
> build_error "error while building test"
> + $CC -lpthread hackbench.c -o hackbench
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.himeno/fuego_test.sh
> b/engine/tests/Benchmark.himeno/fuego_test.sh
> index debce2f..5ad2429 100755
> --- a/engine/tests/Benchmark.himeno/fuego_test.sh
> +++ b/engine/tests/Benchmark.himeno/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=himeno.tar.bz2
>
> function test_build {
> CFLAGS+=" -O3"
> - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch test_suite_ready ||
> build_error "error while building test"
> + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.iperf/fuego_test.sh
> b/engine/tests/Benchmark.iperf/fuego_test.sh
> index 74b4437..ad19052 100755
> --- a/engine/tests/Benchmark.iperf/fuego_test.sh
> +++ b/engine/tests/Benchmark.iperf/fuego_test.sh
> @@ -5,7 +5,7 @@ function test_build {
> make config.h
> sed -i -e "s/#define HAVE_MALLOC 0/#define HAVE_MALLOC 1/g" -e
> "s/#define malloc rpl_malloc/\/\* #undef malloc \*\//g" config.h
> sed -i -e '/HEADERS\(\)/ a\#include "gnu_getopt.h"' src/Settings.cpp
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.linpack/fuego_test.sh
> b/engine/tests/Benchmark.linpack/fuego_test.sh
> index a42d665..98ac835 100755
> --- a/engine/tests/Benchmark.linpack/fuego_test.sh
> +++ b/engine/tests/Benchmark.linpack/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=linpack.tar.bz2
>
> function test_build {
> patch -p0 -N -s < $TEST_HOME/linpack.c.patch
> - $CC $CFLAGS -O -lm -o linpack linpack.c && touch test_suite_ready ||
> build_error "error while building test"
> + $CC $CFLAGS -O -lm -o linpack linpack.c
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.lmbench2/fuego_test.sh
> b/engine/tests/Benchmark.lmbench2/fuego_test.sh
> index 67d4afe..7dd56ca 100755
> --- a/engine/tests/Benchmark.lmbench2/fuego_test.sh
> +++ b/engine/tests/Benchmark.lmbench2/fuego_test.sh
> @@ -11,7 +11,7 @@ function test_build {
> patch -p0 < $TEST_HOME/bench.h.patch
> cd ..
> CFLAGS+=" -g -O"
> - make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch
> test_suite_ready || build_error "error while building test"
> + make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> index 650b71f..d4a8336 100755
> --- a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> +++ b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> @@ -4,7 +4,7 @@ function test_build {
> patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
> rm -f pointer.h && touch pointer.h
> CFLAGS+=" -s -static -Wall -O3"
> - make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready ||
> build_error "error while building test"
> + make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> index 42b78d1..2561951 100755
> --- a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> +++ b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> @@ -4,7 +4,7 @@ function test_build {
> patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
> rm -f pointer.h && touch pointer.h
> CFLAGS+=" -s -static -Wall -O3"
> - make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX
> CPP=$CPP CXXCPP=$CXXCPP && touch test_suite_ready || build_error
> "error while building test"
> + make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX
> CPP=$CPP CXXCPP=$CXXCPP
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.netpipe/fuego_test.sh
> b/engine/tests/Benchmark.netpipe/fuego_test.sh
> index 732982e..a2f1a79 100755
> --- a/engine/tests/Benchmark.netpipe/fuego_test.sh
> +++ b/engine/tests/Benchmark.netpipe/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=NetPIPE-3.7.1.tar.gz
>
> function test_build {
> patch -p1 -N -s < ../../tarballs/netpipe-makefile.patch
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> touch test_suite_ready || build_error "error while building test"
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.signaltest/fuego_test.sh
> b/engine/tests/Benchmark.signaltest/fuego_test.sh
> index 7c0e3fb..fc57c23 100755
> --- a/engine/tests/Benchmark.signaltest/fuego_test.sh
> +++ b/engine/tests/Benchmark.signaltest/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=signaltest.tar.gz
>
> function test_build {
> - make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS" &&
> touch test_suite_ready || build_error "error while building test"
> + make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.tiobench/fuego_test.sh
> b/engine/tests/Benchmark.tiobench/fuego_test.sh
> index a974859..f916335 100755
> --- a/engine/tests/Benchmark.tiobench/fuego_test.sh
> +++ b/engine/tests/Benchmark.tiobench/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=tiobench-0.3.3.tar.gz
>
> function test_build {
> patch -N -s -p1 < $TEST_HOME/tiobench-fix-conflicting-types.patch
> - make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}" && touch
> test_suite_ready || build_error "error while building test"
> + make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Benchmark.x11perf/fuego_test.sh
> b/engine/tests/Benchmark.x11perf/fuego_test.sh
> index b706d2b..70209c1 100755
> --- a/engine/tests/Benchmark.x11perf/fuego_test.sh
> +++ b/engine/tests/Benchmark.x11perf/fuego_test.sh
> @@ -8,7 +8,7 @@ function test_build {
>
> ./configure $CONFIGURE_FLAGS X11PERF_LIBS="-lX11 -lXmu" # force to
> use libXmu instead of libXmuu
> sed -i 's#$(SED)#sed#' Makefile # this is wrong, but somewhy $(SED) in
> Makefile expands to nothing
> - make && touch test_suite_ready
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.LTP/fuego_test.sh
> b/engine/tests/Functional.LTP/fuego_test.sh
> index db5c4d8..f7c9ced 100755
> --- a/engine/tests/Functional.LTP/fuego_test.sh
> +++ b/engine/tests/Functional.LTP/fuego_test.sh
> @@ -89,8 +89,6 @@ function test_build {
>
> cp --parents testcases/realtime/scripts/setenv.sh `pwd`/target_bin/
> cp $TEST_HOME/ltp_target_run.sh `pwd`/target_bin/
> -
> - touch test_suite_ready
> else
> echo "Skipping LTP build"
> fi
> diff --git a/engine/tests/Functional.aiostress/fuego_test.sh
> b/engine/tests/Functional.aiostress/fuego_test.sh
> index 2bebb67..7418af9 100755
> --- a/engine/tests/Functional.aiostress/fuego_test.sh
> +++ b/engine/tests/Functional.aiostress/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=aiostress.tar.gz
>
> function test_build {
> - $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio
> aiostress.c -o aiostress && touch test_suite_ready || build_error "error
> while building test"
> + $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio
> aiostress.c -o aiostress
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.bsdiff/fuego_test.sh
> b/engine/tests/Functional.bsdiff/fuego_test.sh
> index 48723c0..b84c259 100755
> --- a/engine/tests/Functional.bsdiff/fuego_test.sh
> +++ b/engine/tests/Functional.bsdiff/fuego_test.sh
> @@ -1,9 +1,5 @@
> tarball=bsdiff.tar.gz
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> put data/*.modified data/*.original $BOARD_TESTDIR/fuego.$TESTDIR/;
> }
> diff --git a/engine/tests/Functional.bzip2/fuego_test.sh
> b/engine/tests/Functional.bzip2/fuego_test.sh
> index f5c75c5..2375b48 100755
> --- a/engine/tests/Functional.bzip2/fuego_test.sh
> +++ b/engine/tests/Functional.bzip2/fuego_test.sh
> @@ -22,7 +22,6 @@ function test_build {
> if cmp sample3.bz2 sample3.rb2; then echo 'TEST-9 OK'; else echo 'TEST-9
> FAILED'; fi;
> if cmp sample1.tst sample1.ref; then echo 'TEST-10 OK'; else echo 'TEST-10
> FAILED'; fi;
> if cmp sample2.tst sample2.ref; then echo 'TEST-11 OK'; else echo 'TEST-11
> FAILED'; fi;" > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.cmt/fuego_test.sh
> b/engine/tests/Functional.cmt/fuego_test.sh
> index 5a0482d..fc662b3 100755
> --- a/engine/tests/Functional.cmt/fuego_test.sh
> +++ b/engine/tests/Functional.cmt/fuego_test.sh
> @@ -1,9 +1,5 @@
> tarball=dung-3.4.25-m2.tar.gz
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> put ./* $BOARD_TESTDIR/fuego.$TESTDIR/
> }
> diff --git a/engine/tests/Functional.crashme/fuego_test.sh
> b/engine/tests/Functional.crashme/fuego_test.sh
> index f0aea34..66fa374 100755
> --- a/engine/tests/Functional.crashme/fuego_test.sh
> +++ b/engine/tests/Functional.crashme/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=crashme_2.4.tar.bz2
>
> function test_build {
> patch -p1 -N -s < $TEST_HOME/crashme_2.4-9.patch
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> touch test_suite_ready || build_error "error while building test"
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.curl/fuego_test.sh
> b/engine/tests/Functional.curl/fuego_test.sh
> index 945e0e2..81059d8 100755
> --- a/engine/tests/Functional.curl/fuego_test.sh
> +++ b/engine/tests/Functional.curl/fuego_test.sh
> @@ -1,7 +1,3 @@
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> true
> }
> diff --git a/engine/tests/Functional.expat/fuego_test.sh
> b/engine/tests/Functional.expat/fuego_test.sh
> index 5a0f4f5..8a84d89 100755
> --- a/engine/tests/Functional.expat/fuego_test.sh
> +++ b/engine/tests/Functional.expat/fuego_test.sh
> @@ -14,8 +14,8 @@ function test_build {
>
> CXXFLAGS='-I. -I./lib -g -O2'
> ./configure --build=`uname -m`-gnu-linux --host="$PREFIX" #CC="$CC"
> AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" 1>/dev/null
> - make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX"
> CXXFLAGS="$CXXFLAGS" tests/runtestspp; 1>/dev/null && touch
> test_suite_ready || build_error "error while building test"
> + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> CXXCPP="$CXXCPP" || return
> + make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX"
> CXXFLAGS="$CXXFLAGS" tests/runtestspp
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.fixesproto/fuego_test.sh
> b/engine/tests/Functional.fixesproto/fuego_test.sh
> index 86919b9..cebd40c 100755
> --- a/engine/tests/Functional.fixesproto/fuego_test.sh
> +++ b/engine/tests/Functional.fixesproto/fuego_test.sh
> @@ -6,8 +6,6 @@ function test_build {
> if [ -f /usr/include/X11/extensions/xfixeswire.h ]; then echo 'TEST-2 OK';
> else echo 'TEST-2 FAIL'; fi;
> if [ -f /usr/lib/pkgconfig/fixesproto.pc ]; then echo 'TEST-3 OK'; else echo
> 'TEST-3 FAIL'; fi;
> " > run-tests.sh
> -
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.fuego_abort/fuego_test.sh
> b/engine/tests/Functional.fuego_abort/fuego_test.sh
> index b7cab8e..84f0a42 100755
> --- a/engine/tests/Functional.fuego_abort/fuego_test.sh
> +++ b/engine/tests/Functional.fuego_abort/fuego_test.sh
> @@ -9,10 +9,6 @@ function test_pre_check {
> echo "This test has pid $$"
> }
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> true
> }
> diff --git a/engine/tests/Functional.fuego_board_check/fuego_test.sh
> b/engine/tests/Functional.fuego_board_check/fuego_test.sh
> index 580d623..972c46e 100755
> --- a/engine/tests/Functional.fuego_board_check/fuego_test.sh
> +++ b/engine/tests/Functional.fuego_board_check/fuego_test.sh
> @@ -2,10 +2,6 @@
>
> tarfile=scan_for_items.sh
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> put $TEST_HOME/scan_for_items.sh $BOARD_TESTDIR/fuego.$TESTDIR/
> }
> diff --git a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> index 80251c9..8834a3a 100755
> --- a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> +++ b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> @@ -14,7 +14,6 @@ function test_pre_check {
>
> function test_build {
> PHASE_STR="${PHASE_STR}:test_build"
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.glib/fuego_test.sh
> b/engine/tests/Functional.glib/fuego_test.sh
> index c346f05..cbeb35f 100755
> --- a/engine/tests/Functional.glib/fuego_test.sh
> +++ b/engine/tests/Functional.glib/fuego_test.sh
> @@ -8,7 +8,7 @@ function test_build {
> ./configure --prefix=`pwd`/build --cache-file=config.cross --host=$HOST --
> target=$HOST --build=`uname -m`-linux-gnu CC="$CC" AR="$AR"
> RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> CXXFLAGS="$CXXFLAGS"
> make 2>&1
> cd tests && make test; cd -
> - cd glib/tests && make test; cd - && touch test_suite_ready || build_error
> "error while building test"
> + cd glib/tests && make test; cd -
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.glibc/fuego_test.sh
> b/engine/tests/Functional.glibc/fuego_test.sh
> index e387900..6e3213c 100755
> --- a/engine/tests/Functional.glibc/fuego_test.sh
> +++ b/engine/tests/Functional.glibc/fuego_test.sh
> @@ -39,7 +39,6 @@ EOF
> then echo 'TEST-14 OK'; else echo 'TEST-14 FAIL'; fi;
> " > run-tests.sh
>
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.hello_world/fuego_test.sh
> b/engine/tests/Functional.hello_world/fuego_test.sh
> index 27a8344..9c9a394 100755
> --- a/engine/tests/Functional.hello_world/fuego_test.sh
> +++ b/engine/tests/Functional.hello_world/fuego_test.sh
> @@ -3,7 +3,7 @@
> tarball=hello-test-1.1.tgz
>
> function test_build {
> - make && touch test_suite_ready || build_error "error while building
> test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.imagemagick/fuego_test.sh
> b/engine/tests/Functional.imagemagick/fuego_test.sh
> index 3f1c69f..b21eb77 100755
> --- a/engine/tests/Functional.imagemagick/fuego_test.sh
> +++ b/engine/tests/Functional.imagemagick/fuego_test.sh
> @@ -44,7 +44,6 @@ function test_build {
> # if ./djpeg -dct int -ppm -outfile testout.ppm testorig.jpg; then echo 'TEST-
> 1 OK'; else echo 'TEST-1 FAIL'; fi;
> # " > run-tests.sh
>
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.ipv6connect/fuego_test.sh
> b/engine/tests/Functional.ipv6connect/fuego_test.sh
> index bcda1f3..c6fdd05 100755
> --- a/engine/tests/Functional.ipv6connect/fuego_test.sh
> +++ b/engine/tests/Functional.ipv6connect/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=ipv6connect.tar.gz
>
> function test_build {
> - make CC="$CC" LD="$LD" && touch test_suite_ready || build_error
> "error while building test"
> + make CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.jpeg/fuego_test.sh
> b/engine/tests/Functional.jpeg/fuego_test.sh
> index 55f34d6..2cbc237 100755
> --- a/engine/tests/Functional.jpeg/fuego_test.sh
> +++ b/engine/tests/Functional.jpeg/fuego_test.sh
> @@ -16,7 +16,6 @@ function test_build {
> if cmp testimg.ppm testoutp.ppm; then echo 'TEST-10 OK'; else echo
> 'TEST-10 FAIL'; fi;
> if cmp testimgp.jpg testoutp.jpg; then echo 'TEST-11 OK'; else echo 'TEST-
> 11 FAIL'; fi;
> if cmp testorig.jpg testoutt.jpg; then echo 'TEST-12 OK'; else echo 'TEST-12
> FAIL'; fi;" > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.libtar/fuego_test.sh
> b/engine/tests/Functional.libtar/fuego_test.sh
> index 06ef34b..2fcc01f 100755
> --- a/engine/tests/Functional.libtar/fuego_test.sh
> +++ b/engine/tests/Functional.libtar/fuego_test.sh
> @@ -22,7 +22,6 @@ function test_build {
> else
> echo 'TEST-3 FAILED'
> fi" > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.linus_stress/fuego_test.sh
> b/engine/tests/Functional.linus_stress/fuego_test.sh
> index b5b2afb..a03debd 100755
> --- a/engine/tests/Functional.linus_stress/fuego_test.sh
> +++ b/engine/tests/Functional.linus_stress/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=linus_stress.tar.gz
>
> function test_build {
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> touch test_suite_ready || build_error "error while building test"
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.lwip/fuego_test.sh
> b/engine/tests/Functional.lwip/fuego_test.sh
> index d5c8c80..17d15af 100755
> --- a/engine/tests/Functional.lwip/fuego_test.sh
> +++ b/engine/tests/Functional.lwip/fuego_test.sh
> @@ -29,7 +29,6 @@ function test_build {
> killall sntp_test
>
> " > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.pi_tests/fuego_test.sh
> b/engine/tests/Functional.pi_tests/fuego_test.sh
> index de6057e..c3240cc 100755
> --- a/engine/tests/Functional.pi_tests/fuego_test.sh
> +++ b/engine/tests/Functional.pi_tests/fuego_test.sh
> @@ -2,7 +2,7 @@ gitrepo=https://github.com/clrkwllms/rt-tests.git
>
> function test_build {
> cd rt-tests/
> - make CC="$CC" LD="$LD" NUMA=0 pi_stress && touch test_suite_ready
> || build_error "error while building test"
> + make CC="$CC" LD="$LD" NUMA=0 pi_stress
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.pixman/fuego_test.sh
> b/engine/tests/Functional.pixman/fuego_test.sh
> index 7999b0d..a8cd6dd 100755
> --- a/engine/tests/Functional.pixman/fuego_test.sh
> +++ b/engine/tests/Functional.pixman/fuego_test.sh
> @@ -190,7 +190,6 @@ function test_build {
> echo 'TEST-30 FAILED'
> fi
> " > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.pppd/fuego_test.sh
> b/engine/tests/Functional.pppd/fuego_test.sh
> index a622f95..d080871 100755
> --- a/engine/tests/Functional.pppd/fuego_test.sh
> +++ b/engine/tests/Functional.pppd/fuego_test.sh
> @@ -22,7 +22,6 @@ function test_build {
> killall ppp_response
> # killall pppd
> " > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.protobuf/fuego_test.sh
> b/engine/tests/Functional.protobuf/fuego_test.sh
> index 57037d3..2c6cfcb 100755
> --- a/engine/tests/Functional.protobuf/fuego_test.sh
> +++ b/engine/tests/Functional.protobuf/fuego_test.sh
> @@ -27,7 +27,6 @@ function test_build {
> else
> echo 'TEST-4 FAILED'
> fi" > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.rmaptest/fuego_test.sh
> b/engine/tests/Functional.rmaptest/fuego_test.sh
> index 69e08b0..84e360b 100755
> --- a/engine/tests/Functional.rmaptest/fuego_test.sh
> +++ b/engine/tests/Functional.rmaptest/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=rmaptest.tar.gz
>
> function test_build {
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> touch test_suite_ready || build_error "error while building test"
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.scifab/fuego_test.sh
> b/engine/tests/Functional.scifab/fuego_test.sh
> index d874964..418e947 100755
> --- a/engine/tests/Functional.scifab/fuego_test.sh
> +++ b/engine/tests/Functional.scifab/fuego_test.sh
> @@ -1,9 +1,5 @@
> tarball=dung-3.4.25-m2.tar.gz
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> put ./* $OSV_HOME/osv.$TESTDIR/
> }
> diff --git a/engine/tests/Functional.scrashme/fuego_test.sh
> b/engine/tests/Functional.scrashme/fuego_test.sh
> index d173e47..aabb779 100755
> --- a/engine/tests/Functional.scrashme/fuego_test.sh
> +++ b/engine/tests/Functional.scrashme/fuego_test.sh
> @@ -1,19 +1,19 @@
> tarball=scrashme.tar.bz2
>
> function test_pre_check {
> - assert_define FUNCTIONAL_SCRASHME_NUM
> - assert_define FUNCTIONAL_SCRASHME_MODE
> + assert_define FUNCTIONAL_SCRASHME_NUM
> + assert_define FUNCTIONAL_SCRASHME_MODE
> }
>
> function test_build {
> patch -p1 -N -s < $TEST_HOME/scrashme-testfix.patch
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> touch test_suite_ready || build_error "error while building test"
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> - put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
> + put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
> }
>
> function test_run {
> - report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --
> mode=$FUNCTIONAL_SCRASHME_MODE -
> N$FUNCTIONAL_SCRASHME_NUM"
> + report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --
> mode=$FUNCTIONAL_SCRASHME_MODE -
> N$FUNCTIONAL_SCRASHME_NUM"
> }
> diff --git a/engine/tests/Functional.sdhi_0/fuego_test.sh
> b/engine/tests/Functional.sdhi_0/fuego_test.sh
> index b290808..b3cee43 100755
> --- a/engine/tests/Functional.sdhi_0/fuego_test.sh
> +++ b/engine/tests/Functional.sdhi_0/fuego_test.sh
> @@ -1,9 +1,5 @@
> tarball=dung-3.4.25-m2.tar.gz
>
> -function test_build {
> - touch test_suite_ready
> -}
> -
> function test_deploy {
> put ./* $OSV_HOME/osv.$TESTDIR/
> }
> diff --git a/engine/tests/Functional.stress/fuego_test.sh
> b/engine/tests/Functional.stress/fuego_test.sh
> index e9c467f..11bcfe1 100755
> --- a/engine/tests/Functional.stress/fuego_test.sh
> +++ b/engine/tests/Functional.stress/fuego_test.sh
> @@ -2,7 +2,7 @@ tarball=stress-1.0.4.tar.gz
>
> function test_build {
> ./configure --host=$HOST --build=`uname -m`-linux-gnu CC="$CC"
> AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> CFLAGS="$CFLAGS";
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.synctest/fuego_test.sh
> b/engine/tests/Functional.synctest/fuego_test.sh
> index aff0820..95d4dda 100755
> --- a/engine/tests/Functional.synctest/fuego_test.sh
> +++ b/engine/tests/Functional.synctest/fuego_test.sh
> @@ -1,7 +1,7 @@
> tarball=synctest.tar.gz
>
> function test_build {
> - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> touch test_suite_ready || build_error "error while building test"
> + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.thrift/fuego_test.sh
> b/engine/tests/Functional.thrift/fuego_test.sh
> index 8310e4e..17a8c11 100755
> --- a/engine/tests/Functional.thrift/fuego_test.sh
> +++ b/engine/tests/Functional.thrift/fuego_test.sh
> @@ -111,7 +111,6 @@ function test_build {
>
> killall CppServer
> " > run-tests.sh
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.xorg-macros/fuego_test.sh
> b/engine/tests/Functional.xorg-macros/fuego_test.sh
> index 95490be..df10fcd 100755
> --- a/engine/tests/Functional.xorg-macros/fuego_test.sh
> +++ b/engine/tests/Functional.xorg-macros/fuego_test.sh
> @@ -6,7 +6,6 @@ function test_build {
> if [ -f /usr/share/pkgconfig/xorg-macros.pc ]; then echo 'TEST-2 OK'; else
> echo 'TEST-2 FAIL'; fi;
> " > run-tests.sh
>
> - touch test_suite_ready
> }
>
> function test_deploy {
> diff --git a/engine/tests/Functional.zlib/fuego_test.sh
> b/engine/tests/Functional.zlib/fuego_test.sh
> index 7615b5d..c794c52 100755
> --- a/engine/tests/Functional.zlib/fuego_test.sh
> +++ b/engine/tests/Functional.zlib/fuego_test.sh
> @@ -4,7 +4,7 @@ function test_build {
> AR=$PREFIX'-ar rc'
> CPP=$PREFIX'-gcc -E'
> CC="$CC" AR="$AR" CPP="$CPP" ./configure --
> includedir=$SDKROOT/usr/include --libdir=$SDKROOT/usr/lib
> - make LDSHARED="$CC" >/dev/null && touch test_suite_ready ||
> build_error "error while building test"
> + make LDSHARED="$CC" >/dev/null
> }
>
> function test_deploy {
> diff --git a/engine/tests/OpenSSL/openssl.sh
> b/engine/tests/OpenSSL/openssl.sh
> index 95848a7..ec062c7 100755
> --- a/engine/tests/OpenSSL/openssl.sh
> +++ b/engine/tests/OpenSSL/openssl.sh
> @@ -51,5 +51,4 @@ function test_build {
> ../util/shlib_wrap.sh ./shatest
> ../util/shlib_wrap.sh ./ssltest' > run-tests.sh
> rm test/fips_aes_data
> - touch test_suite_ready || build_error "error while building test"
> }
> diff --git a/engine/tests/netperf/netperf.sh
> b/engine/tests/netperf/netperf.sh
> index 2cd6da4..8c14cff 100755
> --- a/engine/tests/netperf/netperf.sh
> +++ b/engine/tests/netperf/netperf.sh
> @@ -1,6 +1,6 @@
> function test_build {
> echo "ac_cv_func_setpgrp_void=yes" > config.cache
> ./configure --build=`./config.guess` --host=$HOST CC="$CC" AR="$AR"
> RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" --config-
> cache
> - make && touch test_suite_ready || build_error "error while building test"
> + make
> }
>
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 05/19] unpack: move code related to spec-defined tarballs to unpack
2017-05-12 8:19 ` [Fuego] [PATCH 05/19] unpack: move code related to spec-defined tarballs to unpack Daniel Sangorrin
@ 2017-05-12 23:05 ` Bird, Timothy
0 siblings, 0 replies; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 23:05 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
Nice change. It makes much more sense in unpack.
Thanks,
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, May 12, 2017 1:20 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 05/19] unpack: move code related to spec-defined
> tarballs to unpack
>
> Test specs can override the default tarball by specifying
> a new one with tarball_name. This code should be inside unpack
> for clarity.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index d9e0b18..9eae52b 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -38,6 +38,16 @@ set -o errtrace
>
> # Unpacks $tarball_path/$tarball into current directory.
> function unpack {
> + # the tarball name can be defined in the spec (tarball_name) or direclty
> + # in fuego_test.sh (tarball). Note that the one in the spec has preference.
> + upName=`echo "${TESTDIR^^}"| tr '.' '_'`
> + spec_tarball="${upName}_TARBALL_NAME"
> +
> + if [ ! -z "${!spec_tarball}" ]; then
> + echo "Using $spec_tarball=${!spec_tarball} from test spec"
> + tarball=${!spec_tarball}
> + fi
> +
> if [ ! -z ${tarball+x} ]; then
> case ${tarball/*./} in
> gz|tgz) key=z ;;
> @@ -170,14 +180,6 @@ function build_error () {
> # Build scripts must call this function in the beginning.
> function pre_build {
> cd ${WORKSPACE}
> - upName=`echo "${JOB_NAME^^}"| tr '.' '_'`
> - spec_tarball="${upName}_TARBALL_NAME"
> -
> - if [ ! -z "${!spec_tarball}" ]
> - then
> - echo "Using $spec_tarball=${!spec_tarball} from test spec"
> - tarball=${!spec_tarball}
> - fi
>
> mkdir -p $TRIPLET && cd $TRIPLET
> if concurrent_check; then
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 07/19] unpack: handle possible errors
2017-05-12 8:19 ` [Fuego] [PATCH 07/19] unpack: handle possible errors Daniel Sangorrin
@ 2017-05-12 23:09 ` Bird, Timothy
0 siblings, 0 replies; 31+ messages in thread
From: Bird, Timothy @ 2017-05-12 23:09 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, May 12, 2017 1:20 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 07/19] unpack: handle possible errors
>
> Tests can use source code from a tarball or from a git repository.
> Differentiate both cases with return sentences and handle
> possible errors.
>
> Note that there are tests that do not require any source code
> so make sure they do not cause an error as well.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 12 ++++++++----
> 1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index 9eae52b..d5bf969 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -36,7 +36,7 @@ trap signal_handler SIGTERM SIGHUP SIGALRM SIGINT
> EXIT
> # cause ERR trap setting to be visible inside functions
> set -o errtrace
>
> -# Unpacks $tarball_path/$tarball into current directory.
> +# Unpacks/clones the test source code into the current directory.
> function unpack {
> # the tarball name can be defined in the spec (tarball_name) or direclty
> # in fuego_test.sh (tarball). Note that the one in the spec has preference.
> @@ -53,9 +53,10 @@ function unpack {
> gz|tgz) key=z ;;
> bz2) key=j ;;
> tar) key= ;;
> - *) echo "Unknown $tarball file format. Not unpacking."; return;;
> + *) echo "Unknown $tarball file format. Not unpacking."; return 1;;
This makes this an abort case, where previously it was just a warning.
I'm OK with it - this makes the base script definition more strict, and
it probably good to promote correct scripts.
> esac
> tar ${key}xf $TEST_HOME/$tarball --strip-components=1
> + return
> fi
>
> if [ ! -z ${gitrepo+x} ]; then
> @@ -64,8 +65,11 @@ function unpack {
> fi
> # FIXTHIS: support commit ids (log and checkout)
> # FIXTHIS: strip components like tarballs
> - git clone --depth=1 --branch=$gitbranch $gitrepo
> + git clone --depth=1 --branch=$gitbranch $gitrepo .
Would be really nice to add a commit checkout here! :-)
(Yes - I see the FIXTHIS above.) It would be really nice
to allow a test spec or a board variable to define the
commit id. I think I might be able to concoct a bisect
functionality for ftc, if the commit id can come from
<board>.vars. But here, I think you just need to
check if GIT_COMMIT_ID is defined, and if so try
to check it out.
> + return
> fi
> +
> + echo "No tarball or gitrepo definition."
> }
>
> function is_empty {
> @@ -191,7 +195,7 @@ function pre_build {
> fi
>
> if [ ! -e test_suite_ready ]; then
> - unpack
> + unpack || abort_job "Error while unpacking"
> return 0
> fi
> fi
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
2017-05-12 23:02 ` Bird, Timothy
@ 2017-05-16 2:35 ` Daniel Sangorrin
2017-05-16 4:35 ` Daniel Sangorrin
0 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-16 2:35 UTC (permalink / raw)
To: 'Bird, Timothy', fuego
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> Sent: Saturday, May 13, 2017 8:03 AM
>
> This one worries me a bit.
>
> I like the simplification, but I'm not sure I trust the logic (see below for a question).
>
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Friday, May 12, 2017 1:20 AM
> >
> > +++ b/engine/tests/Benchmark.Dhrystone/fuego_test.sh
> > @@ -5,14 +5,14 @@ function test_pre_check {
> > }
> >
> > function test_build {
> > - patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return 1
> > + patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return
>
> Does this end up returning the return value from patch?
Yes, return does not alter the $? variable as far as I know.
> How does this interact with 'set -e'? I think this should work
> since bash doesn't trap command errors when the command
> is in a logic expression. But does an error trap from the
> non-zero return from test_build?
Before this patch, jobs were calling the "build_error" which
ended up aborting the job.
Now, the return value of test_build is handled in the function "build"
using a logic expression (no error trap expected) which calls
aboart_job in case of error. So the way it works hasn't really changed.
set -e will still trap errors in test_build that do not have a "|| return" expression.
One thing we might need to add is a target cleanup before calling the abort function.
But I think this should not be necessary. We should instead move part of the code
in pre_test to the pre_deploy function (which by the way I forgot to update!!).
Thanks,
Daniel
> > CFLAGS+=" -DTIME"
> > LDFLAGS+=" -lm"
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch
> > test_suite_ready || return 1
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
> > }
> >
> > function test_deploy {
> > - put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/ || return 1
> > + put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/
> > }
> >
> > function test_run {
> > diff --git a/engine/tests/Benchmark.GLMark/fuego_test.sh
> > b/engine/tests/Benchmark.GLMark/fuego_test.sh
> > index f409109..7deb6e2 100755
> > --- a/engine/tests/Benchmark.GLMark/fuego_test.sh
> > +++ b/engine/tests/Benchmark.GLMark/fuego_test.sh
> > @@ -7,7 +7,7 @@ function test_build {
> > -L${SDKROOT}/usr/lib -Wl,-rpath-link=${SDKROOT}/usr/lib \
> > -L${SDKROOT}/lib \
> > *.cpp -o glmark -lSDL -lGL \
> > - -lGLU -lGLEW && touch test_suite_ready || build_error "error while
> > building test"
> > + -lGLU -lGLEW
> > # -Wl,--allow-shlib-undefined *.cpp -o glmark -lSDL -lGL \
> > }
> >
> > diff --git a/engine/tests/Benchmark.IOzone/fuego_test.sh
> > b/engine/tests/Benchmark.IOzone/fuego_test.sh
> > index fe06e75..16b5c61 100755
> > --- a/engine/tests/Benchmark.IOzone/fuego_test.sh
> > +++ b/engine/tests/Benchmark.IOzone/fuego_test.sh
> > @@ -11,10 +11,10 @@ function test_build {
> > TARGET=linux-AMD64
> > else
> > echo "platform based on $ARCHITECTURE is not supported by
> > benchmark"
> > - build_error "error while building test"
> > + return 1
> > fi
> >
> > - make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready ||
> > build_error "error while building test"
> > + make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.Interbench/fuego_test.sh
> > b/engine/tests/Benchmark.Interbench/fuego_test.sh
> > index 5f950e2..31e6d2f 100755
> > --- a/engine/tests/Benchmark.Interbench/fuego_test.sh
> > +++ b/engine/tests/Benchmark.Interbench/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=interbench-0.31.tar.bz2
> >
> > function test_build {
> > patch -p0 < $TEST_HOME/interbench.c.patch
> > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while
> > building test"
> > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.Java/fuego_test.sh
> > b/engine/tests/Benchmark.Java/fuego_test.sh
> > index 7978263..22cdbb7 100755
> > --- a/engine/tests/Benchmark.Java/fuego_test.sh
> > +++ b/engine/tests/Benchmark.Java/fuego_test.sh
> > @@ -1,9 +1,5 @@
> > tarball=java_perf.tar
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > put *.jar $BOARD_TESTDIR/fuego.$TESTDIR/
> > }
> > diff --git a/engine/tests/Benchmark.Stream/fuego_test.sh
> > b/engine/tests/Benchmark.Stream/fuego_test.sh
> > index ddeb26c..9f4c04c 100755
> > --- a/engine/tests/Benchmark.Stream/fuego_test.sh
> > +++ b/engine/tests/Benchmark.Stream/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=stream.tar.bz2
> >
> > function test_build {
> > - make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR"
> > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch
> > test_suite_ready || build_error "error while building test"
> > + make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR"
> > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > b/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > index 925bfcb..5e791f0 100755
> > --- a/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > +++ b/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=Whetstone.tar.bz2
> >
> > function test_build {
> > CFLAGS+=" -DTIME"
> > - make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> > LIBS=" -lm" && touch test_suite_ready || build_error "error while building
> > test"
> > + make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> > LIBS=" -lm"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.aim7/fuego_test.sh
> > b/engine/tests/Benchmark.aim7/fuego_test.sh
> > index 2cee552..87f9df9 100755
> > --- a/engine/tests/Benchmark.aim7/fuego_test.sh
> > +++ b/engine/tests/Benchmark.aim7/fuego_test.sh
> > @@ -3,7 +3,7 @@ tarball=osdl-aim-7.0.1.13.tar.gz
> > function test_build {
> > ./bootstrap
> > PKG_CONFIG_PATH=${SDKROOT}/usr/lib/pkgconfig
> > PKG_CONFIG_ALLOW_SYSTEM_LIBS=1
> > PKG_CONFIG_SYSROOT_DIR=${SDKROOT} ./configure --host=$HOST --
> > build=`uname -m`-linux-gnu LDFLAGS=-L${SDKROOT}/usr/lib CPPFLAGS=-
> > I${SDKROOT}/usr/include CFLAGS=-I${SDKROOT}/usr/include LIBS=-laio --
> > prefix=$BOARD_TESTDIR/$TESTDIR --
> > datarootdir=$BOARD_TESTDIR/$TESTDIR
> > - make && touch test_suite_ready || build_error "error while building
> > test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > b/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > index ed5273d..f849946 100755
> > --- a/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > +++ b/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > @@ -5,7 +5,7 @@ function test_build {
> > patch -p0 -N -s < $TEST_HOME/blobsallad.auto.patch
> > patch -p0 -N -s < $TEST_HOME/bs_main.c.patch
> >
> > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" SDKROOT="$SDKROOT" && touch test_suite_ready ||
> > build_error "error while building test"
> > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" SDKROOT="$SDKROOT"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.bonnie/fuego_test.sh
> > b/engine/tests/Benchmark.bonnie/fuego_test.sh
> > index 562088a..d94dce4 100755
> > --- a/engine/tests/Benchmark.bonnie/fuego_test.sh
> > +++ b/engine/tests/Benchmark.bonnie/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=bonnie++-1.03e.tar.gz
> >
> > function test_build {
> > ./configure --host=$HOST --build=`uname -m`-linux-gnu;
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > b/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > index 657de9b..2631a82 100755
> > --- a/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > +++ b/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=cyclictest.tar.gz
> >
> > function test_build {
> > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch
> > test_suite_ready || build_error "error while building test"
> > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.dbench/fuego_test.sh
> > b/engine/tests/Benchmark.dbench/fuego_test.sh
> > index d84c820..f7160f9 100755
> > --- a/engine/tests/Benchmark.dbench/fuego_test.sh
> > +++ b/engine/tests/Benchmark.dbench/fuego_test.sh
> > @@ -3,7 +3,7 @@ tarball=dbench-3.04.tar.gz
> > function test_build {
> > patch -N -s -p1 < $TEST_HOME/dbench_startup.patch
> > ./configure --host=$HOST --build=`uname -m`-linux-gnu
> > CFLAGS="$CFLAGS";
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > b/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > index 1e1e7ac..28b9fc0 100755
> > --- a/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > +++ b/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=ebizzy-0.3.tar.gz
> >
> > function test_build {
> > - $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c && touch
> > test_suite_ready || build_error "error while building test"
> > + $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.ffsb/fuego_test.sh
> > b/engine/tests/Benchmark.ffsb/fuego_test.sh
> > index 2037c8f..80a157d 100755
> > --- a/engine/tests/Benchmark.ffsb/fuego_test.sh
> > +++ b/engine/tests/Benchmark.ffsb/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=ffsb-6.0-rc2.tar.bz2
> >
> > function test_build {
> > ./configure --host=$HOST --build=`uname -m`-linux-gnu CC=$CC AR=$AR
> > RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP
> > CFLAGS="$CFLAGS";
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.fio/fuego_test.sh
> > b/engine/tests/Benchmark.fio/fuego_test.sh
> > index 344e445..039ec15 100755
> > --- a/engine/tests/Benchmark.fio/fuego_test.sh
> > +++ b/engine/tests/Benchmark.fio/fuego_test.sh
> > @@ -14,7 +14,6 @@ function test_build {
> > # >/dev/null
> >
> > make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP"
> > - touch ../test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > index 562b659..80e4999 100644
> > --- a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > +++ b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > @@ -1,7 +1,3 @@
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > return 0
> > }
> > diff --git a/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > b/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > index 07a4487..45b07bb 100755
> > --- a/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > +++ b/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > @@ -9,7 +9,7 @@ function test_build {
> > cd ..
> > export PATH=/usr/local/bin:$PATH
> > ./configure --prefix=$BOARD_TESTDIR/$TESTDIR --host=$HOST --
> > build=`uname -m`-linux-gnu --target=$HOST
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.hackbench/fuego_test.sh
> > b/engine/tests/Benchmark.hackbench/fuego_test.sh
> > index 3b45dd3..385f533 100755
> > --- a/engine/tests/Benchmark.hackbench/fuego_test.sh
> > +++ b/engine/tests/Benchmark.hackbench/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=hackbench.tar.gz
> >
> > function test_build {
> > - $CC -lpthread hackbench.c -o hackbench && touch test_suite_ready ||
> > build_error "error while building test"
> > + $CC -lpthread hackbench.c -o hackbench
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.himeno/fuego_test.sh
> > b/engine/tests/Benchmark.himeno/fuego_test.sh
> > index debce2f..5ad2429 100755
> > --- a/engine/tests/Benchmark.himeno/fuego_test.sh
> > +++ b/engine/tests/Benchmark.himeno/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=himeno.tar.bz2
> >
> > function test_build {
> > CFLAGS+=" -O3"
> > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch test_suite_ready ||
> > build_error "error while building test"
> > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.iperf/fuego_test.sh
> > b/engine/tests/Benchmark.iperf/fuego_test.sh
> > index 74b4437..ad19052 100755
> > --- a/engine/tests/Benchmark.iperf/fuego_test.sh
> > +++ b/engine/tests/Benchmark.iperf/fuego_test.sh
> > @@ -5,7 +5,7 @@ function test_build {
> > make config.h
> > sed -i -e "s/#define HAVE_MALLOC 0/#define HAVE_MALLOC 1/g" -e
> > "s/#define malloc rpl_malloc/\/\* #undef malloc \*\//g" config.h
> > sed -i -e '/HEADERS\(\)/ a\#include "gnu_getopt.h"' src/Settings.cpp
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.linpack/fuego_test.sh
> > b/engine/tests/Benchmark.linpack/fuego_test.sh
> > index a42d665..98ac835 100755
> > --- a/engine/tests/Benchmark.linpack/fuego_test.sh
> > +++ b/engine/tests/Benchmark.linpack/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=linpack.tar.bz2
> >
> > function test_build {
> > patch -p0 -N -s < $TEST_HOME/linpack.c.patch
> > - $CC $CFLAGS -O -lm -o linpack linpack.c && touch test_suite_ready ||
> > build_error "error while building test"
> > + $CC $CFLAGS -O -lm -o linpack linpack.c
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > b/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > index 67d4afe..7dd56ca 100755
> > --- a/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > +++ b/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > @@ -11,7 +11,7 @@ function test_build {
> > patch -p0 < $TEST_HOME/bench.h.patch
> > cd ..
> > CFLAGS+=" -g -O"
> > - make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch
> > test_suite_ready || build_error "error while building test"
> > + make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > index 650b71f..d4a8336 100755
> > --- a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > +++ b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > @@ -4,7 +4,7 @@ function test_build {
> > patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
> > rm -f pointer.h && touch pointer.h
> > CFLAGS+=" -s -static -Wall -O3"
> > - make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready ||
> > build_error "error while building test"
> > + make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > index 42b78d1..2561951 100755
> > --- a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > +++ b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > @@ -4,7 +4,7 @@ function test_build {
> > patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
> > rm -f pointer.h && touch pointer.h
> > CFLAGS+=" -s -static -Wall -O3"
> > - make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX
> > CPP=$CPP CXXCPP=$CXXCPP && touch test_suite_ready || build_error
> > "error while building test"
> > + make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX
> > CPP=$CPP CXXCPP=$CXXCPP
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.netpipe/fuego_test.sh
> > b/engine/tests/Benchmark.netpipe/fuego_test.sh
> > index 732982e..a2f1a79 100755
> > --- a/engine/tests/Benchmark.netpipe/fuego_test.sh
> > +++ b/engine/tests/Benchmark.netpipe/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=NetPIPE-3.7.1.tar.gz
> >
> > function test_build {
> > patch -p1 -N -s < ../../tarballs/netpipe-makefile.patch
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.signaltest/fuego_test.sh
> > b/engine/tests/Benchmark.signaltest/fuego_test.sh
> > index 7c0e3fb..fc57c23 100755
> > --- a/engine/tests/Benchmark.signaltest/fuego_test.sh
> > +++ b/engine/tests/Benchmark.signaltest/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=signaltest.tar.gz
> >
> > function test_build {
> > - make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.tiobench/fuego_test.sh
> > b/engine/tests/Benchmark.tiobench/fuego_test.sh
> > index a974859..f916335 100755
> > --- a/engine/tests/Benchmark.tiobench/fuego_test.sh
> > +++ b/engine/tests/Benchmark.tiobench/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=tiobench-0.3.3.tar.gz
> >
> > function test_build {
> > patch -N -s -p1 < $TEST_HOME/tiobench-fix-conflicting-types.patch
> > - make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}" && touch
> > test_suite_ready || build_error "error while building test"
> > + make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Benchmark.x11perf/fuego_test.sh
> > b/engine/tests/Benchmark.x11perf/fuego_test.sh
> > index b706d2b..70209c1 100755
> > --- a/engine/tests/Benchmark.x11perf/fuego_test.sh
> > +++ b/engine/tests/Benchmark.x11perf/fuego_test.sh
> > @@ -8,7 +8,7 @@ function test_build {
> >
> > ./configure $CONFIGURE_FLAGS X11PERF_LIBS="-lX11 -lXmu" # force to
> > use libXmu instead of libXmuu
> > sed -i 's#$(SED)#sed#' Makefile # this is wrong, but somewhy $(SED) in
> > Makefile expands to nothing
> > - make && touch test_suite_ready
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.LTP/fuego_test.sh
> > b/engine/tests/Functional.LTP/fuego_test.sh
> > index db5c4d8..f7c9ced 100755
> > --- a/engine/tests/Functional.LTP/fuego_test.sh
> > +++ b/engine/tests/Functional.LTP/fuego_test.sh
> > @@ -89,8 +89,6 @@ function test_build {
> >
> > cp --parents testcases/realtime/scripts/setenv.sh `pwd`/target_bin/
> > cp $TEST_HOME/ltp_target_run.sh `pwd`/target_bin/
> > -
> > - touch test_suite_ready
> > else
> > echo "Skipping LTP build"
> > fi
> > diff --git a/engine/tests/Functional.aiostress/fuego_test.sh
> > b/engine/tests/Functional.aiostress/fuego_test.sh
> > index 2bebb67..7418af9 100755
> > --- a/engine/tests/Functional.aiostress/fuego_test.sh
> > +++ b/engine/tests/Functional.aiostress/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=aiostress.tar.gz
> >
> > function test_build {
> > - $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio
> > aiostress.c -o aiostress && touch test_suite_ready || build_error "error
> > while building test"
> > + $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio
> > aiostress.c -o aiostress
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.bsdiff/fuego_test.sh
> > b/engine/tests/Functional.bsdiff/fuego_test.sh
> > index 48723c0..b84c259 100755
> > --- a/engine/tests/Functional.bsdiff/fuego_test.sh
> > +++ b/engine/tests/Functional.bsdiff/fuego_test.sh
> > @@ -1,9 +1,5 @@
> > tarball=bsdiff.tar.gz
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > put data/*.modified data/*.original $BOARD_TESTDIR/fuego.$TESTDIR/;
> > }
> > diff --git a/engine/tests/Functional.bzip2/fuego_test.sh
> > b/engine/tests/Functional.bzip2/fuego_test.sh
> > index f5c75c5..2375b48 100755
> > --- a/engine/tests/Functional.bzip2/fuego_test.sh
> > +++ b/engine/tests/Functional.bzip2/fuego_test.sh
> > @@ -22,7 +22,6 @@ function test_build {
> > if cmp sample3.bz2 sample3.rb2; then echo 'TEST-9 OK'; else echo 'TEST-9
> > FAILED'; fi;
> > if cmp sample1.tst sample1.ref; then echo 'TEST-10 OK'; else echo 'TEST-10
> > FAILED'; fi;
> > if cmp sample2.tst sample2.ref; then echo 'TEST-11 OK'; else echo 'TEST-11
> > FAILED'; fi;" > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.cmt/fuego_test.sh
> > b/engine/tests/Functional.cmt/fuego_test.sh
> > index 5a0482d..fc662b3 100755
> > --- a/engine/tests/Functional.cmt/fuego_test.sh
> > +++ b/engine/tests/Functional.cmt/fuego_test.sh
> > @@ -1,9 +1,5 @@
> > tarball=dung-3.4.25-m2.tar.gz
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > put ./* $BOARD_TESTDIR/fuego.$TESTDIR/
> > }
> > diff --git a/engine/tests/Functional.crashme/fuego_test.sh
> > b/engine/tests/Functional.crashme/fuego_test.sh
> > index f0aea34..66fa374 100755
> > --- a/engine/tests/Functional.crashme/fuego_test.sh
> > +++ b/engine/tests/Functional.crashme/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=crashme_2.4.tar.bz2
> >
> > function test_build {
> > patch -p1 -N -s < $TEST_HOME/crashme_2.4-9.patch
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.curl/fuego_test.sh
> > b/engine/tests/Functional.curl/fuego_test.sh
> > index 945e0e2..81059d8 100755
> > --- a/engine/tests/Functional.curl/fuego_test.sh
> > +++ b/engine/tests/Functional.curl/fuego_test.sh
> > @@ -1,7 +1,3 @@
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > true
> > }
> > diff --git a/engine/tests/Functional.expat/fuego_test.sh
> > b/engine/tests/Functional.expat/fuego_test.sh
> > index 5a0f4f5..8a84d89 100755
> > --- a/engine/tests/Functional.expat/fuego_test.sh
> > +++ b/engine/tests/Functional.expat/fuego_test.sh
> > @@ -14,8 +14,8 @@ function test_build {
> >
> > CXXFLAGS='-I. -I./lib -g -O2'
> > ./configure --build=`uname -m`-gnu-linux --host="$PREFIX" #CC="$CC"
> > AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" 1>/dev/null
> > - make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX"
> > CXXFLAGS="$CXXFLAGS" tests/runtestspp; 1>/dev/null && touch
> > test_suite_ready || build_error "error while building test"
> > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > CXXCPP="$CXXCPP" || return
> > + make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX"
> > CXXFLAGS="$CXXFLAGS" tests/runtestspp
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.fixesproto/fuego_test.sh
> > b/engine/tests/Functional.fixesproto/fuego_test.sh
> > index 86919b9..cebd40c 100755
> > --- a/engine/tests/Functional.fixesproto/fuego_test.sh
> > +++ b/engine/tests/Functional.fixesproto/fuego_test.sh
> > @@ -6,8 +6,6 @@ function test_build {
> > if [ -f /usr/include/X11/extensions/xfixeswire.h ]; then echo 'TEST-2 OK';
> > else echo 'TEST-2 FAIL'; fi;
> > if [ -f /usr/lib/pkgconfig/fixesproto.pc ]; then echo 'TEST-3 OK'; else echo
> > 'TEST-3 FAIL'; fi;
> > " > run-tests.sh
> > -
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.fuego_abort/fuego_test.sh
> > b/engine/tests/Functional.fuego_abort/fuego_test.sh
> > index b7cab8e..84f0a42 100755
> > --- a/engine/tests/Functional.fuego_abort/fuego_test.sh
> > +++ b/engine/tests/Functional.fuego_abort/fuego_test.sh
> > @@ -9,10 +9,6 @@ function test_pre_check {
> > echo "This test has pid $$"
> > }
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > true
> > }
> > diff --git a/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > b/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > index 580d623..972c46e 100755
> > --- a/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > +++ b/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > @@ -2,10 +2,6 @@
> >
> > tarfile=scan_for_items.sh
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > put $TEST_HOME/scan_for_items.sh $BOARD_TESTDIR/fuego.$TESTDIR/
> > }
> > diff --git a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > index 80251c9..8834a3a 100755
> > --- a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > +++ b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > @@ -14,7 +14,6 @@ function test_pre_check {
> >
> > function test_build {
> > PHASE_STR="${PHASE_STR}:test_build"
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.glib/fuego_test.sh
> > b/engine/tests/Functional.glib/fuego_test.sh
> > index c346f05..cbeb35f 100755
> > --- a/engine/tests/Functional.glib/fuego_test.sh
> > +++ b/engine/tests/Functional.glib/fuego_test.sh
> > @@ -8,7 +8,7 @@ function test_build {
> > ./configure --prefix=`pwd`/build --cache-file=config.cross --host=$HOST --
> > target=$HOST --build=`uname -m`-linux-gnu CC="$CC" AR="$AR"
> > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > CXXFLAGS="$CXXFLAGS"
> > make 2>&1
> > cd tests && make test; cd -
> > - cd glib/tests && make test; cd - && touch test_suite_ready || build_error
> > "error while building test"
> > + cd glib/tests && make test; cd -
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.glibc/fuego_test.sh
> > b/engine/tests/Functional.glibc/fuego_test.sh
> > index e387900..6e3213c 100755
> > --- a/engine/tests/Functional.glibc/fuego_test.sh
> > +++ b/engine/tests/Functional.glibc/fuego_test.sh
> > @@ -39,7 +39,6 @@ EOF
> > then echo 'TEST-14 OK'; else echo 'TEST-14 FAIL'; fi;
> > " > run-tests.sh
> >
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.hello_world/fuego_test.sh
> > b/engine/tests/Functional.hello_world/fuego_test.sh
> > index 27a8344..9c9a394 100755
> > --- a/engine/tests/Functional.hello_world/fuego_test.sh
> > +++ b/engine/tests/Functional.hello_world/fuego_test.sh
> > @@ -3,7 +3,7 @@
> > tarball=hello-test-1.1.tgz
> >
> > function test_build {
> > - make && touch test_suite_ready || build_error "error while building
> > test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.imagemagick/fuego_test.sh
> > b/engine/tests/Functional.imagemagick/fuego_test.sh
> > index 3f1c69f..b21eb77 100755
> > --- a/engine/tests/Functional.imagemagick/fuego_test.sh
> > +++ b/engine/tests/Functional.imagemagick/fuego_test.sh
> > @@ -44,7 +44,6 @@ function test_build {
> > # if ./djpeg -dct int -ppm -outfile testout.ppm testorig.jpg; then echo 'TEST-
> > 1 OK'; else echo 'TEST-1 FAIL'; fi;
> > # " > run-tests.sh
> >
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.ipv6connect/fuego_test.sh
> > b/engine/tests/Functional.ipv6connect/fuego_test.sh
> > index bcda1f3..c6fdd05 100755
> > --- a/engine/tests/Functional.ipv6connect/fuego_test.sh
> > +++ b/engine/tests/Functional.ipv6connect/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=ipv6connect.tar.gz
> >
> > function test_build {
> > - make CC="$CC" LD="$LD" && touch test_suite_ready || build_error
> > "error while building test"
> > + make CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.jpeg/fuego_test.sh
> > b/engine/tests/Functional.jpeg/fuego_test.sh
> > index 55f34d6..2cbc237 100755
> > --- a/engine/tests/Functional.jpeg/fuego_test.sh
> > +++ b/engine/tests/Functional.jpeg/fuego_test.sh
> > @@ -16,7 +16,6 @@ function test_build {
> > if cmp testimg.ppm testoutp.ppm; then echo 'TEST-10 OK'; else echo
> > 'TEST-10 FAIL'; fi;
> > if cmp testimgp.jpg testoutp.jpg; then echo 'TEST-11 OK'; else echo 'TEST-
> > 11 FAIL'; fi;
> > if cmp testorig.jpg testoutt.jpg; then echo 'TEST-12 OK'; else echo 'TEST-12
> > FAIL'; fi;" > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.libtar/fuego_test.sh
> > b/engine/tests/Functional.libtar/fuego_test.sh
> > index 06ef34b..2fcc01f 100755
> > --- a/engine/tests/Functional.libtar/fuego_test.sh
> > +++ b/engine/tests/Functional.libtar/fuego_test.sh
> > @@ -22,7 +22,6 @@ function test_build {
> > else
> > echo 'TEST-3 FAILED'
> > fi" > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.linus_stress/fuego_test.sh
> > b/engine/tests/Functional.linus_stress/fuego_test.sh
> > index b5b2afb..a03debd 100755
> > --- a/engine/tests/Functional.linus_stress/fuego_test.sh
> > +++ b/engine/tests/Functional.linus_stress/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=linus_stress.tar.gz
> >
> > function test_build {
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.lwip/fuego_test.sh
> > b/engine/tests/Functional.lwip/fuego_test.sh
> > index d5c8c80..17d15af 100755
> > --- a/engine/tests/Functional.lwip/fuego_test.sh
> > +++ b/engine/tests/Functional.lwip/fuego_test.sh
> > @@ -29,7 +29,6 @@ function test_build {
> > killall sntp_test
> >
> > " > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.pi_tests/fuego_test.sh
> > b/engine/tests/Functional.pi_tests/fuego_test.sh
> > index de6057e..c3240cc 100755
> > --- a/engine/tests/Functional.pi_tests/fuego_test.sh
> > +++ b/engine/tests/Functional.pi_tests/fuego_test.sh
> > @@ -2,7 +2,7 @@ gitrepo=https://github.com/clrkwllms/rt-tests.git
> >
> > function test_build {
> > cd rt-tests/
> > - make CC="$CC" LD="$LD" NUMA=0 pi_stress && touch test_suite_ready
> > || build_error "error while building test"
> > + make CC="$CC" LD="$LD" NUMA=0 pi_stress
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.pixman/fuego_test.sh
> > b/engine/tests/Functional.pixman/fuego_test.sh
> > index 7999b0d..a8cd6dd 100755
> > --- a/engine/tests/Functional.pixman/fuego_test.sh
> > +++ b/engine/tests/Functional.pixman/fuego_test.sh
> > @@ -190,7 +190,6 @@ function test_build {
> > echo 'TEST-30 FAILED'
> > fi
> > " > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.pppd/fuego_test.sh
> > b/engine/tests/Functional.pppd/fuego_test.sh
> > index a622f95..d080871 100755
> > --- a/engine/tests/Functional.pppd/fuego_test.sh
> > +++ b/engine/tests/Functional.pppd/fuego_test.sh
> > @@ -22,7 +22,6 @@ function test_build {
> > killall ppp_response
> > # killall pppd
> > " > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.protobuf/fuego_test.sh
> > b/engine/tests/Functional.protobuf/fuego_test.sh
> > index 57037d3..2c6cfcb 100755
> > --- a/engine/tests/Functional.protobuf/fuego_test.sh
> > +++ b/engine/tests/Functional.protobuf/fuego_test.sh
> > @@ -27,7 +27,6 @@ function test_build {
> > else
> > echo 'TEST-4 FAILED'
> > fi" > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.rmaptest/fuego_test.sh
> > b/engine/tests/Functional.rmaptest/fuego_test.sh
> > index 69e08b0..84e360b 100755
> > --- a/engine/tests/Functional.rmaptest/fuego_test.sh
> > +++ b/engine/tests/Functional.rmaptest/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=rmaptest.tar.gz
> >
> > function test_build {
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.scifab/fuego_test.sh
> > b/engine/tests/Functional.scifab/fuego_test.sh
> > index d874964..418e947 100755
> > --- a/engine/tests/Functional.scifab/fuego_test.sh
> > +++ b/engine/tests/Functional.scifab/fuego_test.sh
> > @@ -1,9 +1,5 @@
> > tarball=dung-3.4.25-m2.tar.gz
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > put ./* $OSV_HOME/osv.$TESTDIR/
> > }
> > diff --git a/engine/tests/Functional.scrashme/fuego_test.sh
> > b/engine/tests/Functional.scrashme/fuego_test.sh
> > index d173e47..aabb779 100755
> > --- a/engine/tests/Functional.scrashme/fuego_test.sh
> > +++ b/engine/tests/Functional.scrashme/fuego_test.sh
> > @@ -1,19 +1,19 @@
> > tarball=scrashme.tar.bz2
> >
> > function test_pre_check {
> > - assert_define FUNCTIONAL_SCRASHME_NUM
> > - assert_define FUNCTIONAL_SCRASHME_MODE
> > + assert_define FUNCTIONAL_SCRASHME_NUM
> > + assert_define FUNCTIONAL_SCRASHME_MODE
> > }
> >
> > function test_build {
> > patch -p1 -N -s < $TEST_HOME/scrashme-testfix.patch
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > - put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
> > + put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
> > }
> >
> > function test_run {
> > - report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --
> > mode=$FUNCTIONAL_SCRASHME_MODE -
> > N$FUNCTIONAL_SCRASHME_NUM"
> > + report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --
> > mode=$FUNCTIONAL_SCRASHME_MODE -
> > N$FUNCTIONAL_SCRASHME_NUM"
> > }
> > diff --git a/engine/tests/Functional.sdhi_0/fuego_test.sh
> > b/engine/tests/Functional.sdhi_0/fuego_test.sh
> > index b290808..b3cee43 100755
> > --- a/engine/tests/Functional.sdhi_0/fuego_test.sh
> > +++ b/engine/tests/Functional.sdhi_0/fuego_test.sh
> > @@ -1,9 +1,5 @@
> > tarball=dung-3.4.25-m2.tar.gz
> >
> > -function test_build {
> > - touch test_suite_ready
> > -}
> > -
> > function test_deploy {
> > put ./* $OSV_HOME/osv.$TESTDIR/
> > }
> > diff --git a/engine/tests/Functional.stress/fuego_test.sh
> > b/engine/tests/Functional.stress/fuego_test.sh
> > index e9c467f..11bcfe1 100755
> > --- a/engine/tests/Functional.stress/fuego_test.sh
> > +++ b/engine/tests/Functional.stress/fuego_test.sh
> > @@ -2,7 +2,7 @@ tarball=stress-1.0.4.tar.gz
> >
> > function test_build {
> > ./configure --host=$HOST --build=`uname -m`-linux-gnu CC="$CC"
> > AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > CFLAGS="$CFLAGS";
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.synctest/fuego_test.sh
> > b/engine/tests/Functional.synctest/fuego_test.sh
> > index aff0820..95d4dda 100755
> > --- a/engine/tests/Functional.synctest/fuego_test.sh
> > +++ b/engine/tests/Functional.synctest/fuego_test.sh
> > @@ -1,7 +1,7 @@
> > tarball=synctest.tar.gz
> >
> > function test_build {
> > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > touch test_suite_ready || build_error "error while building test"
> > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.thrift/fuego_test.sh
> > b/engine/tests/Functional.thrift/fuego_test.sh
> > index 8310e4e..17a8c11 100755
> > --- a/engine/tests/Functional.thrift/fuego_test.sh
> > +++ b/engine/tests/Functional.thrift/fuego_test.sh
> > @@ -111,7 +111,6 @@ function test_build {
> >
> > killall CppServer
> > " > run-tests.sh
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.xorg-macros/fuego_test.sh
> > b/engine/tests/Functional.xorg-macros/fuego_test.sh
> > index 95490be..df10fcd 100755
> > --- a/engine/tests/Functional.xorg-macros/fuego_test.sh
> > +++ b/engine/tests/Functional.xorg-macros/fuego_test.sh
> > @@ -6,7 +6,6 @@ function test_build {
> > if [ -f /usr/share/pkgconfig/xorg-macros.pc ]; then echo 'TEST-2 OK'; else
> > echo 'TEST-2 FAIL'; fi;
> > " > run-tests.sh
> >
> > - touch test_suite_ready
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/Functional.zlib/fuego_test.sh
> > b/engine/tests/Functional.zlib/fuego_test.sh
> > index 7615b5d..c794c52 100755
> > --- a/engine/tests/Functional.zlib/fuego_test.sh
> > +++ b/engine/tests/Functional.zlib/fuego_test.sh
> > @@ -4,7 +4,7 @@ function test_build {
> > AR=$PREFIX'-ar rc'
> > CPP=$PREFIX'-gcc -E'
> > CC="$CC" AR="$AR" CPP="$CPP" ./configure --
> > includedir=$SDKROOT/usr/include --libdir=$SDKROOT/usr/lib
> > - make LDSHARED="$CC" >/dev/null && touch test_suite_ready ||
> > build_error "error while building test"
> > + make LDSHARED="$CC" >/dev/null
> > }
> >
> > function test_deploy {
> > diff --git a/engine/tests/OpenSSL/openssl.sh
> > b/engine/tests/OpenSSL/openssl.sh
> > index 95848a7..ec062c7 100755
> > --- a/engine/tests/OpenSSL/openssl.sh
> > +++ b/engine/tests/OpenSSL/openssl.sh
> > @@ -51,5 +51,4 @@ function test_build {
> > ../util/shlib_wrap.sh ./shatest
> > ../util/shlib_wrap.sh ./ssltest' > run-tests.sh
> > rm test/fips_aes_data
> > - touch test_suite_ready || build_error "error while building test"
> > }
> > diff --git a/engine/tests/netperf/netperf.sh
> > b/engine/tests/netperf/netperf.sh
> > index 2cd6da4..8c14cff 100755
> > --- a/engine/tests/netperf/netperf.sh
> > +++ b/engine/tests/netperf/netperf.sh
> > @@ -1,6 +1,6 @@
> > function test_build {
> > echo "ac_cv_func_setpgrp_void=yes" > config.cache
> > ./configure --build=`./config.guess` --host=$HOST CC="$CC" AR="$AR"
> > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" --config-
> > cache
> > - make && touch test_suite_ready || build_error "error while building test"
> > + make
> > }
> >
> > --
> > 2.7.4
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
2017-05-16 2:35 ` Daniel Sangorrin
@ 2017-05-16 4:35 ` Daniel Sangorrin
2017-05-16 4:52 ` Bird, Timothy
0 siblings, 1 reply; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-16 4:35 UTC (permalink / raw)
To: 'Bird, Timothy', fuego
Hi Tim,
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Tuesday, May 16, 2017 11:35 AM
> > > function test_build {
> > > - patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return 1
> > > + patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return
> >
> > Does this end up returning the return value from patch?
>
> Yes, return does not alter the $? variable as far as I know.
>
> > How does this interact with 'set -e'? I think this should work
> > since bash doesn't trap command errors when the command
> > is in a logic expression. But does an error trap from the
> > non-zero return from test_build?
>
> Before this patch, jobs were calling the "build_error" which
> ended up aborting the job.
>
> Now, the return value of test_build is handled in the function "build"
> using a logic expression (no error trap expected) which calls
> aboart_job in case of error. So the way it works hasn't really changed.
>
> set -e will still trap errors in test_build that do not have a "|| return" expression.
I double checked this, an set -e will NOT trap them because the build has a || expression.
Therefore test writers need to use "|| return" to handle errors within test_build.
One alternative is to change
call_if_present test_build && ret=0 || ret=$?
for
call_if_present test_build
ret=$?
In this case, errors in test_build will be trap and marked as failure by the shell.
Should I change it?
Thanks,
Daniel
> One thing we might need to add is a target cleanup before calling the abort function.
> But I think this should not be necessary. We should instead move part of the code
> in pre_test to the pre_deploy function (which by the way I forgot to update!!).
>
> Thanks,
> Daniel
>
>
> > > CFLAGS+=" -DTIME"
> > > LDFLAGS+=" -lm"
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch
> > > test_suite_ready || return 1
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
> > > }
> > >
> > > function test_deploy {
> > > - put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/ || return 1
> > > + put dhrystone $BOARD_TESTDIR/fuego.$TESTDIR/
> > > }
> > >
> > > function test_run {
> > > diff --git a/engine/tests/Benchmark.GLMark/fuego_test.sh
> > > b/engine/tests/Benchmark.GLMark/fuego_test.sh
> > > index f409109..7deb6e2 100755
> > > --- a/engine/tests/Benchmark.GLMark/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.GLMark/fuego_test.sh
> > > @@ -7,7 +7,7 @@ function test_build {
> > > -L${SDKROOT}/usr/lib -Wl,-rpath-link=${SDKROOT}/usr/lib \
> > > -L${SDKROOT}/lib \
> > > *.cpp -o glmark -lSDL -lGL \
> > > - -lGLU -lGLEW && touch test_suite_ready || build_error "error while
> > > building test"
> > > + -lGLU -lGLEW
> > > # -Wl,--allow-shlib-undefined *.cpp -o glmark -lSDL -lGL \
> > > }
> > >
> > > diff --git a/engine/tests/Benchmark.IOzone/fuego_test.sh
> > > b/engine/tests/Benchmark.IOzone/fuego_test.sh
> > > index fe06e75..16b5c61 100755
> > > --- a/engine/tests/Benchmark.IOzone/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.IOzone/fuego_test.sh
> > > @@ -11,10 +11,10 @@ function test_build {
> > > TARGET=linux-AMD64
> > > else
> > > echo "platform based on $ARCHITECTURE is not supported by
> > > benchmark"
> > > - build_error "error while building test"
> > > + return 1
> > > fi
> > >
> > > - make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + make $TARGET GCC="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.Interbench/fuego_test.sh
> > > b/engine/tests/Benchmark.Interbench/fuego_test.sh
> > > index 5f950e2..31e6d2f 100755
> > > --- a/engine/tests/Benchmark.Interbench/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.Interbench/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=interbench-0.31.tar.bz2
> > >
> > > function test_build {
> > > patch -p0 < $TEST_HOME/interbench.c.patch
> > > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" && touch test_suite_ready || build_error "error while
> > > building test"
> > > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.Java/fuego_test.sh
> > > b/engine/tests/Benchmark.Java/fuego_test.sh
> > > index 7978263..22cdbb7 100755
> > > --- a/engine/tests/Benchmark.Java/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.Java/fuego_test.sh
> > > @@ -1,9 +1,5 @@
> > > tarball=java_perf.tar
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > put *.jar $BOARD_TESTDIR/fuego.$TESTDIR/
> > > }
> > > diff --git a/engine/tests/Benchmark.Stream/fuego_test.sh
> > > b/engine/tests/Benchmark.Stream/fuego_test.sh
> > > index ddeb26c..9f4c04c 100755
> > > --- a/engine/tests/Benchmark.Stream/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.Stream/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=stream.tar.bz2
> > >
> > > function test_build {
> > > - make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR"
> > > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch
> > > test_suite_ready || build_error "error while building test"
> > > + make stream_c.exe CFLAGS+="${CFLAGS}" CC="$CC" AR="$AR"
> > > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > > b/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > > index 925bfcb..5e791f0 100755
> > > --- a/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.Whetstone/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=Whetstone.tar.bz2
> > >
> > > function test_build {
> > > CFLAGS+=" -DTIME"
> > > - make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> > > LIBS=" -lm" && touch test_suite_ready || build_error "error while building
> > > test"
> > > + make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> > > LIBS=" -lm"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.aim7/fuego_test.sh
> > > b/engine/tests/Benchmark.aim7/fuego_test.sh
> > > index 2cee552..87f9df9 100755
> > > --- a/engine/tests/Benchmark.aim7/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.aim7/fuego_test.sh
> > > @@ -3,7 +3,7 @@ tarball=osdl-aim-7.0.1.13.tar.gz
> > > function test_build {
> > > ./bootstrap
> > > PKG_CONFIG_PATH=${SDKROOT}/usr/lib/pkgconfig
> > > PKG_CONFIG_ALLOW_SYSTEM_LIBS=1
> > > PKG_CONFIG_SYSROOT_DIR=${SDKROOT} ./configure --host=$HOST --
> > > build=`uname -m`-linux-gnu LDFLAGS=-L${SDKROOT}/usr/lib CPPFLAGS=-
> > > I${SDKROOT}/usr/include CFLAGS=-I${SDKROOT}/usr/include LIBS=-laio --
> > > prefix=$BOARD_TESTDIR/$TESTDIR --
> > > datarootdir=$BOARD_TESTDIR/$TESTDIR
> > > - make && touch test_suite_ready || build_error "error while building
> > > test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > > b/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > > index ed5273d..f849946 100755
> > > --- a/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.blobsallad/fuego_test.sh
> > > @@ -5,7 +5,7 @@ function test_build {
> > > patch -p0 -N -s < $TEST_HOME/blobsallad.auto.patch
> > > patch -p0 -N -s < $TEST_HOME/bs_main.c.patch
> > >
> > > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" SDKROOT="$SDKROOT" && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" SDKROOT="$SDKROOT"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.bonnie/fuego_test.sh
> > > b/engine/tests/Benchmark.bonnie/fuego_test.sh
> > > index 562088a..d94dce4 100755
> > > --- a/engine/tests/Benchmark.bonnie/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.bonnie/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=bonnie++-1.03e.tar.gz
> > >
> > > function test_build {
> > > ./configure --host=$HOST --build=`uname -m`-linux-gnu;
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > > b/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > > index 657de9b..2631a82 100755
> > > --- a/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.cyclictest/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=cyclictest.tar.gz
> > >
> > > function test_build {
> > > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" && touch
> > > test_suite_ready || build_error "error while building test"
> > > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.dbench/fuego_test.sh
> > > b/engine/tests/Benchmark.dbench/fuego_test.sh
> > > index d84c820..f7160f9 100755
> > > --- a/engine/tests/Benchmark.dbench/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.dbench/fuego_test.sh
> > > @@ -3,7 +3,7 @@ tarball=dbench-3.04.tar.gz
> > > function test_build {
> > > patch -N -s -p1 < $TEST_HOME/dbench_startup.patch
> > > ./configure --host=$HOST --build=`uname -m`-linux-gnu
> > > CFLAGS="$CFLAGS";
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > > b/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > > index 1e1e7ac..28b9fc0 100755
> > > --- a/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.ebizzy/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=ebizzy-0.3.tar.gz
> > >
> > > function test_build {
> > > - $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c && touch
> > > test_suite_ready || build_error "error while building test"
> > > + $CC -Wall -Wshadow -lpthread -o ebizzy ebizzy.c
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.ffsb/fuego_test.sh
> > > b/engine/tests/Benchmark.ffsb/fuego_test.sh
> > > index 2037c8f..80a157d 100755
> > > --- a/engine/tests/Benchmark.ffsb/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.ffsb/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=ffsb-6.0-rc2.tar.bz2
> > >
> > > function test_build {
> > > ./configure --host=$HOST --build=`uname -m`-linux-gnu CC=$CC AR=$AR
> > > RANLIB=$RANLIB CXX=$CXX CPP=$CPP CXXCPP=$CXXCPP
> > > CFLAGS="$CFLAGS";
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.fio/fuego_test.sh
> > > b/engine/tests/Benchmark.fio/fuego_test.sh
> > > index 344e445..039ec15 100755
> > > --- a/engine/tests/Benchmark.fio/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.fio/fuego_test.sh
> > > @@ -14,7 +14,6 @@ function test_build {
> > > # >/dev/null
> > >
> > > make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP"
> > > - touch ../test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > > b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > > index 562b659..80e4999 100644
> > > --- a/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.fuego_check_plots/fuego_test.sh
> > > @@ -1,7 +1,3 @@
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > return 0
> > > }
> > > diff --git a/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > > b/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > > index 07a4487..45b07bb 100755
> > > --- a/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.gtkperf/fuego_test.sh
> > > @@ -9,7 +9,7 @@ function test_build {
> > > cd ..
> > > export PATH=/usr/local/bin:$PATH
> > > ./configure --prefix=$BOARD_TESTDIR/$TESTDIR --host=$HOST --
> > > build=`uname -m`-linux-gnu --target=$HOST
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.hackbench/fuego_test.sh
> > > b/engine/tests/Benchmark.hackbench/fuego_test.sh
> > > index 3b45dd3..385f533 100755
> > > --- a/engine/tests/Benchmark.hackbench/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.hackbench/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=hackbench.tar.gz
> > >
> > > function test_build {
> > > - $CC -lpthread hackbench.c -o hackbench && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + $CC -lpthread hackbench.c -o hackbench
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.himeno/fuego_test.sh
> > > b/engine/tests/Benchmark.himeno/fuego_test.sh
> > > index debce2f..5ad2429 100755
> > > --- a/engine/tests/Benchmark.himeno/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.himeno/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=himeno.tar.bz2
> > >
> > > function test_build {
> > > CFLAGS+=" -O3"
> > > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.iperf/fuego_test.sh
> > > b/engine/tests/Benchmark.iperf/fuego_test.sh
> > > index 74b4437..ad19052 100755
> > > --- a/engine/tests/Benchmark.iperf/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.iperf/fuego_test.sh
> > > @@ -5,7 +5,7 @@ function test_build {
> > > make config.h
> > > sed -i -e "s/#define HAVE_MALLOC 0/#define HAVE_MALLOC 1/g" -e
> > > "s/#define malloc rpl_malloc/\/\* #undef malloc \*\//g" config.h
> > > sed -i -e '/HEADERS\(\)/ a\#include "gnu_getopt.h"' src/Settings.cpp
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.linpack/fuego_test.sh
> > > b/engine/tests/Benchmark.linpack/fuego_test.sh
> > > index a42d665..98ac835 100755
> > > --- a/engine/tests/Benchmark.linpack/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.linpack/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=linpack.tar.bz2
> > >
> > > function test_build {
> > > patch -p0 -N -s < $TEST_HOME/linpack.c.patch
> > > - $CC $CFLAGS -O -lm -o linpack linpack.c && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + $CC $CFLAGS -O -lm -o linpack linpack.c
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > > b/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > > index 67d4afe..7dd56ca 100755
> > > --- a/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.lmbench2/fuego_test.sh
> > > @@ -11,7 +11,7 @@ function test_build {
> > > patch -p0 < $TEST_HOME/bench.h.patch
> > > cd ..
> > > CFLAGS+=" -g -O"
> > > - make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS" && touch
> > > test_suite_ready || build_error "error while building test"
> > > + make OS="$PREFIX" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS="$CFLAGS"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > > b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > > index 650b71f..d4a8336 100755
> > > --- a/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.nbench-byte/fuego_test.sh
> > > @@ -4,7 +4,7 @@ function test_build {
> > > patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
> > > rm -f pointer.h && touch pointer.h
> > > CFLAGS+=" -s -static -Wall -O3"
> > > - make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + make CFLAGS="${CFLAGS}" CC="$CC" AR="$AR" RANLIB="$RANLIB"
> > > CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > > b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > > index 42b78d1..2561951 100755
> > > --- a/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.nbench_byte/fuego_test.sh
> > > @@ -4,7 +4,7 @@ function test_build {
> > > patch -N -s -p0 < $TEST_HOME/nbench.Makefile.patch
> > > rm -f pointer.h && touch pointer.h
> > > CFLAGS+=" -s -static -Wall -O3"
> > > - make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX
> > > CPP=$CPP CXXCPP=$CXXCPP && touch test_suite_ready || build_error
> > > "error while building test"
> > > + make CFLAGS="${CFLAGS}" CC=$CC AR=$AR RANLIB=$RANLIB CXX=$CXX
> > > CPP=$CPP CXXCPP=$CXXCPP
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.netpipe/fuego_test.sh
> > > b/engine/tests/Benchmark.netpipe/fuego_test.sh
> > > index 732982e..a2f1a79 100755
> > > --- a/engine/tests/Benchmark.netpipe/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.netpipe/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=NetPIPE-3.7.1.tar.gz
> > >
> > > function test_build {
> > > patch -p1 -N -s < ../../tarballs/netpipe-makefile.patch
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.signaltest/fuego_test.sh
> > > b/engine/tests/Benchmark.signaltest/fuego_test.sh
> > > index 7c0e3fb..fc57c23 100755
> > > --- a/engine/tests/Benchmark.signaltest/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.signaltest/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=signaltest.tar.gz
> > >
> > > function test_build {
> > > - make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.tiobench/fuego_test.sh
> > > b/engine/tests/Benchmark.tiobench/fuego_test.sh
> > > index a974859..f916335 100755
> > > --- a/engine/tests/Benchmark.tiobench/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.tiobench/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=tiobench-0.3.3.tar.gz
> > >
> > > function test_build {
> > > patch -N -s -p1 < $TEST_HOME/tiobench-fix-conflicting-types.patch
> > > - make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}" && touch
> > > test_suite_ready || build_error "error while building test"
> > > + make LINK="$CC" CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX"
> > > CPP="$CPP" CXXCPP="$CXXCPP" CFLAGS+="${CFLAGS}"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Benchmark.x11perf/fuego_test.sh
> > > b/engine/tests/Benchmark.x11perf/fuego_test.sh
> > > index b706d2b..70209c1 100755
> > > --- a/engine/tests/Benchmark.x11perf/fuego_test.sh
> > > +++ b/engine/tests/Benchmark.x11perf/fuego_test.sh
> > > @@ -8,7 +8,7 @@ function test_build {
> > >
> > > ./configure $CONFIGURE_FLAGS X11PERF_LIBS="-lX11 -lXmu" # force to
> > > use libXmu instead of libXmuu
> > > sed -i 's#$(SED)#sed#' Makefile # this is wrong, but somewhy $(SED) in
> > > Makefile expands to nothing
> > > - make && touch test_suite_ready
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.LTP/fuego_test.sh
> > > b/engine/tests/Functional.LTP/fuego_test.sh
> > > index db5c4d8..f7c9ced 100755
> > > --- a/engine/tests/Functional.LTP/fuego_test.sh
> > > +++ b/engine/tests/Functional.LTP/fuego_test.sh
> > > @@ -89,8 +89,6 @@ function test_build {
> > >
> > > cp --parents testcases/realtime/scripts/setenv.sh `pwd`/target_bin/
> > > cp $TEST_HOME/ltp_target_run.sh `pwd`/target_bin/
> > > -
> > > - touch test_suite_ready
> > > else
> > > echo "Skipping LTP build"
> > > fi
> > > diff --git a/engine/tests/Functional.aiostress/fuego_test.sh
> > > b/engine/tests/Functional.aiostress/fuego_test.sh
> > > index 2bebb67..7418af9 100755
> > > --- a/engine/tests/Functional.aiostress/fuego_test.sh
> > > +++ b/engine/tests/Functional.aiostress/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=aiostress.tar.gz
> > >
> > > function test_build {
> > > - $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio
> > > aiostress.c -o aiostress && touch test_suite_ready || build_error "error
> > > while building test"
> > > + $CC -I $SDKROOT/usr/include -L $SDKROOT/usr/lib -Wall -lpthread -laio
> > > aiostress.c -o aiostress
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.bsdiff/fuego_test.sh
> > > b/engine/tests/Functional.bsdiff/fuego_test.sh
> > > index 48723c0..b84c259 100755
> > > --- a/engine/tests/Functional.bsdiff/fuego_test.sh
> > > +++ b/engine/tests/Functional.bsdiff/fuego_test.sh
> > > @@ -1,9 +1,5 @@
> > > tarball=bsdiff.tar.gz
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > put data/*.modified data/*.original $BOARD_TESTDIR/fuego.$TESTDIR/;
> > > }
> > > diff --git a/engine/tests/Functional.bzip2/fuego_test.sh
> > > b/engine/tests/Functional.bzip2/fuego_test.sh
> > > index f5c75c5..2375b48 100755
> > > --- a/engine/tests/Functional.bzip2/fuego_test.sh
> > > +++ b/engine/tests/Functional.bzip2/fuego_test.sh
> > > @@ -22,7 +22,6 @@ function test_build {
> > > if cmp sample3.bz2 sample3.rb2; then echo 'TEST-9 OK'; else echo 'TEST-9
> > > FAILED'; fi;
> > > if cmp sample1.tst sample1.ref; then echo 'TEST-10 OK'; else echo 'TEST-10
> > > FAILED'; fi;
> > > if cmp sample2.tst sample2.ref; then echo 'TEST-11 OK'; else echo 'TEST-11
> > > FAILED'; fi;" > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.cmt/fuego_test.sh
> > > b/engine/tests/Functional.cmt/fuego_test.sh
> > > index 5a0482d..fc662b3 100755
> > > --- a/engine/tests/Functional.cmt/fuego_test.sh
> > > +++ b/engine/tests/Functional.cmt/fuego_test.sh
> > > @@ -1,9 +1,5 @@
> > > tarball=dung-3.4.25-m2.tar.gz
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > put ./* $BOARD_TESTDIR/fuego.$TESTDIR/
> > > }
> > > diff --git a/engine/tests/Functional.crashme/fuego_test.sh
> > > b/engine/tests/Functional.crashme/fuego_test.sh
> > > index f0aea34..66fa374 100755
> > > --- a/engine/tests/Functional.crashme/fuego_test.sh
> > > +++ b/engine/tests/Functional.crashme/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=crashme_2.4.tar.bz2
> > >
> > > function test_build {
> > > patch -p1 -N -s < $TEST_HOME/crashme_2.4-9.patch
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.curl/fuego_test.sh
> > > b/engine/tests/Functional.curl/fuego_test.sh
> > > index 945e0e2..81059d8 100755
> > > --- a/engine/tests/Functional.curl/fuego_test.sh
> > > +++ b/engine/tests/Functional.curl/fuego_test.sh
> > > @@ -1,7 +1,3 @@
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > true
> > > }
> > > diff --git a/engine/tests/Functional.expat/fuego_test.sh
> > > b/engine/tests/Functional.expat/fuego_test.sh
> > > index 5a0f4f5..8a84d89 100755
> > > --- a/engine/tests/Functional.expat/fuego_test.sh
> > > +++ b/engine/tests/Functional.expat/fuego_test.sh
> > > @@ -14,8 +14,8 @@ function test_build {
> > >
> > > CXXFLAGS='-I. -I./lib -g -O2'
> > > ./configure --build=`uname -m`-gnu-linux --host="$PREFIX" #CC="$CC"
> > > AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > > - make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" 1>/dev/null
> > > - make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX"
> > > CXXFLAGS="$CXXFLAGS" tests/runtestspp; 1>/dev/null && touch
> > > test_suite_ready || build_error "error while building test"
> > > + make CC="$CC" AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP"
> > > CXXCPP="$CXXCPP" || return
> > > + make CFLAGS="$CFLAGS" CC="$CC" CXX=$PREFIX-g++ CXX="$CXX"
> > > CXXFLAGS="$CXXFLAGS" tests/runtestspp
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.fixesproto/fuego_test.sh
> > > b/engine/tests/Functional.fixesproto/fuego_test.sh
> > > index 86919b9..cebd40c 100755
> > > --- a/engine/tests/Functional.fixesproto/fuego_test.sh
> > > +++ b/engine/tests/Functional.fixesproto/fuego_test.sh
> > > @@ -6,8 +6,6 @@ function test_build {
> > > if [ -f /usr/include/X11/extensions/xfixeswire.h ]; then echo 'TEST-2 OK';
> > > else echo 'TEST-2 FAIL'; fi;
> > > if [ -f /usr/lib/pkgconfig/fixesproto.pc ]; then echo 'TEST-3 OK'; else echo
> > > 'TEST-3 FAIL'; fi;
> > > " > run-tests.sh
> > > -
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.fuego_abort/fuego_test.sh
> > > b/engine/tests/Functional.fuego_abort/fuego_test.sh
> > > index b7cab8e..84f0a42 100755
> > > --- a/engine/tests/Functional.fuego_abort/fuego_test.sh
> > > +++ b/engine/tests/Functional.fuego_abort/fuego_test.sh
> > > @@ -9,10 +9,6 @@ function test_pre_check {
> > > echo "This test has pid $$"
> > > }
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > true
> > > }
> > > diff --git a/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > > b/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > > index 580d623..972c46e 100755
> > > --- a/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > > +++ b/engine/tests/Functional.fuego_board_check/fuego_test.sh
> > > @@ -2,10 +2,6 @@
> > >
> > > tarfile=scan_for_items.sh
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > put $TEST_HOME/scan_for_items.sh $BOARD_TESTDIR/fuego.$TESTDIR/
> > > }
> > > diff --git a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > > b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > > index 80251c9..8834a3a 100755
> > > --- a/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > > +++ b/engine/tests/Functional.fuego_test_phases/fuego_test.sh
> > > @@ -14,7 +14,6 @@ function test_pre_check {
> > >
> > > function test_build {
> > > PHASE_STR="${PHASE_STR}:test_build"
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.glib/fuego_test.sh
> > > b/engine/tests/Functional.glib/fuego_test.sh
> > > index c346f05..cbeb35f 100755
> > > --- a/engine/tests/Functional.glib/fuego_test.sh
> > > +++ b/engine/tests/Functional.glib/fuego_test.sh
> > > @@ -8,7 +8,7 @@ function test_build {
> > > ./configure --prefix=`pwd`/build --cache-file=config.cross --host=$HOST --
> > > target=$HOST --build=`uname -m`-linux-gnu CC="$CC" AR="$AR"
> > > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > > CXXFLAGS="$CXXFLAGS"
> > > make 2>&1
> > > cd tests && make test; cd -
> > > - cd glib/tests && make test; cd - && touch test_suite_ready || build_error
> > > "error while building test"
> > > + cd glib/tests && make test; cd -
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.glibc/fuego_test.sh
> > > b/engine/tests/Functional.glibc/fuego_test.sh
> > > index e387900..6e3213c 100755
> > > --- a/engine/tests/Functional.glibc/fuego_test.sh
> > > +++ b/engine/tests/Functional.glibc/fuego_test.sh
> > > @@ -39,7 +39,6 @@ EOF
> > > then echo 'TEST-14 OK'; else echo 'TEST-14 FAIL'; fi;
> > > " > run-tests.sh
> > >
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.hello_world/fuego_test.sh
> > > b/engine/tests/Functional.hello_world/fuego_test.sh
> > > index 27a8344..9c9a394 100755
> > > --- a/engine/tests/Functional.hello_world/fuego_test.sh
> > > +++ b/engine/tests/Functional.hello_world/fuego_test.sh
> > > @@ -3,7 +3,7 @@
> > > tarball=hello-test-1.1.tgz
> > >
> > > function test_build {
> > > - make && touch test_suite_ready || build_error "error while building
> > > test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.imagemagick/fuego_test.sh
> > > b/engine/tests/Functional.imagemagick/fuego_test.sh
> > > index 3f1c69f..b21eb77 100755
> > > --- a/engine/tests/Functional.imagemagick/fuego_test.sh
> > > +++ b/engine/tests/Functional.imagemagick/fuego_test.sh
> > > @@ -44,7 +44,6 @@ function test_build {
> > > # if ./djpeg -dct int -ppm -outfile testout.ppm testorig.jpg; then echo 'TEST-
> > > 1 OK'; else echo 'TEST-1 FAIL'; fi;
> > > # " > run-tests.sh
> > >
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.ipv6connect/fuego_test.sh
> > > b/engine/tests/Functional.ipv6connect/fuego_test.sh
> > > index bcda1f3..c6fdd05 100755
> > > --- a/engine/tests/Functional.ipv6connect/fuego_test.sh
> > > +++ b/engine/tests/Functional.ipv6connect/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=ipv6connect.tar.gz
> > >
> > > function test_build {
> > > - make CC="$CC" LD="$LD" && touch test_suite_ready || build_error
> > > "error while building test"
> > > + make CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.jpeg/fuego_test.sh
> > > b/engine/tests/Functional.jpeg/fuego_test.sh
> > > index 55f34d6..2cbc237 100755
> > > --- a/engine/tests/Functional.jpeg/fuego_test.sh
> > > +++ b/engine/tests/Functional.jpeg/fuego_test.sh
> > > @@ -16,7 +16,6 @@ function test_build {
> > > if cmp testimg.ppm testoutp.ppm; then echo 'TEST-10 OK'; else echo
> > > 'TEST-10 FAIL'; fi;
> > > if cmp testimgp.jpg testoutp.jpg; then echo 'TEST-11 OK'; else echo 'TEST-
> > > 11 FAIL'; fi;
> > > if cmp testorig.jpg testoutt.jpg; then echo 'TEST-12 OK'; else echo 'TEST-12
> > > FAIL'; fi;" > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.libtar/fuego_test.sh
> > > b/engine/tests/Functional.libtar/fuego_test.sh
> > > index 06ef34b..2fcc01f 100755
> > > --- a/engine/tests/Functional.libtar/fuego_test.sh
> > > +++ b/engine/tests/Functional.libtar/fuego_test.sh
> > > @@ -22,7 +22,6 @@ function test_build {
> > > else
> > > echo 'TEST-3 FAILED'
> > > fi" > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.linus_stress/fuego_test.sh
> > > b/engine/tests/Functional.linus_stress/fuego_test.sh
> > > index b5b2afb..a03debd 100755
> > > --- a/engine/tests/Functional.linus_stress/fuego_test.sh
> > > +++ b/engine/tests/Functional.linus_stress/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=linus_stress.tar.gz
> > >
> > > function test_build {
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.lwip/fuego_test.sh
> > > b/engine/tests/Functional.lwip/fuego_test.sh
> > > index d5c8c80..17d15af 100755
> > > --- a/engine/tests/Functional.lwip/fuego_test.sh
> > > +++ b/engine/tests/Functional.lwip/fuego_test.sh
> > > @@ -29,7 +29,6 @@ function test_build {
> > > killall sntp_test
> > >
> > > " > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.pi_tests/fuego_test.sh
> > > b/engine/tests/Functional.pi_tests/fuego_test.sh
> > > index de6057e..c3240cc 100755
> > > --- a/engine/tests/Functional.pi_tests/fuego_test.sh
> > > +++ b/engine/tests/Functional.pi_tests/fuego_test.sh
> > > @@ -2,7 +2,7 @@ gitrepo=https://github.com/clrkwllms/rt-tests.git
> > >
> > > function test_build {
> > > cd rt-tests/
> > > - make CC="$CC" LD="$LD" NUMA=0 pi_stress && touch test_suite_ready
> > > || build_error "error while building test"
> > > + make CC="$CC" LD="$LD" NUMA=0 pi_stress
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.pixman/fuego_test.sh
> > > b/engine/tests/Functional.pixman/fuego_test.sh
> > > index 7999b0d..a8cd6dd 100755
> > > --- a/engine/tests/Functional.pixman/fuego_test.sh
> > > +++ b/engine/tests/Functional.pixman/fuego_test.sh
> > > @@ -190,7 +190,6 @@ function test_build {
> > > echo 'TEST-30 FAILED'
> > > fi
> > > " > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.pppd/fuego_test.sh
> > > b/engine/tests/Functional.pppd/fuego_test.sh
> > > index a622f95..d080871 100755
> > > --- a/engine/tests/Functional.pppd/fuego_test.sh
> > > +++ b/engine/tests/Functional.pppd/fuego_test.sh
> > > @@ -22,7 +22,6 @@ function test_build {
> > > killall ppp_response
> > > # killall pppd
> > > " > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.protobuf/fuego_test.sh
> > > b/engine/tests/Functional.protobuf/fuego_test.sh
> > > index 57037d3..2c6cfcb 100755
> > > --- a/engine/tests/Functional.protobuf/fuego_test.sh
> > > +++ b/engine/tests/Functional.protobuf/fuego_test.sh
> > > @@ -27,7 +27,6 @@ function test_build {
> > > else
> > > echo 'TEST-4 FAILED'
> > > fi" > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.rmaptest/fuego_test.sh
> > > b/engine/tests/Functional.rmaptest/fuego_test.sh
> > > index 69e08b0..84e360b 100755
> > > --- a/engine/tests/Functional.rmaptest/fuego_test.sh
> > > +++ b/engine/tests/Functional.rmaptest/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=rmaptest.tar.gz
> > >
> > > function test_build {
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.scifab/fuego_test.sh
> > > b/engine/tests/Functional.scifab/fuego_test.sh
> > > index d874964..418e947 100755
> > > --- a/engine/tests/Functional.scifab/fuego_test.sh
> > > +++ b/engine/tests/Functional.scifab/fuego_test.sh
> > > @@ -1,9 +1,5 @@
> > > tarball=dung-3.4.25-m2.tar.gz
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > put ./* $OSV_HOME/osv.$TESTDIR/
> > > }
> > > diff --git a/engine/tests/Functional.scrashme/fuego_test.sh
> > > b/engine/tests/Functional.scrashme/fuego_test.sh
> > > index d173e47..aabb779 100755
> > > --- a/engine/tests/Functional.scrashme/fuego_test.sh
> > > +++ b/engine/tests/Functional.scrashme/fuego_test.sh
> > > @@ -1,19 +1,19 @@
> > > tarball=scrashme.tar.bz2
> > >
> > > function test_pre_check {
> > > - assert_define FUNCTIONAL_SCRASHME_NUM
> > > - assert_define FUNCTIONAL_SCRASHME_MODE
> > > + assert_define FUNCTIONAL_SCRASHME_NUM
> > > + assert_define FUNCTIONAL_SCRASHME_MODE
> > > }
> > >
> > > function test_build {
> > > patch -p1 -N -s < $TEST_HOME/scrashme-testfix.patch
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > - put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
> > > + put scrashme $BOARD_TESTDIR/fuego.$TESTDIR/
> > > }
> > >
> > > function test_run {
> > > - report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --
> > > mode=$FUNCTIONAL_SCRASHME_MODE -
> > > N$FUNCTIONAL_SCRASHME_NUM"
> > > + report "cd $BOARD_TESTDIR/fuego.$TESTDIR; ./scrashme --
> > > mode=$FUNCTIONAL_SCRASHME_MODE -
> > > N$FUNCTIONAL_SCRASHME_NUM"
> > > }
> > > diff --git a/engine/tests/Functional.sdhi_0/fuego_test.sh
> > > b/engine/tests/Functional.sdhi_0/fuego_test.sh
> > > index b290808..b3cee43 100755
> > > --- a/engine/tests/Functional.sdhi_0/fuego_test.sh
> > > +++ b/engine/tests/Functional.sdhi_0/fuego_test.sh
> > > @@ -1,9 +1,5 @@
> > > tarball=dung-3.4.25-m2.tar.gz
> > >
> > > -function test_build {
> > > - touch test_suite_ready
> > > -}
> > > -
> > > function test_deploy {
> > > put ./* $OSV_HOME/osv.$TESTDIR/
> > > }
> > > diff --git a/engine/tests/Functional.stress/fuego_test.sh
> > > b/engine/tests/Functional.stress/fuego_test.sh
> > > index e9c467f..11bcfe1 100755
> > > --- a/engine/tests/Functional.stress/fuego_test.sh
> > > +++ b/engine/tests/Functional.stress/fuego_test.sh
> > > @@ -2,7 +2,7 @@ tarball=stress-1.0.4.tar.gz
> > >
> > > function test_build {
> > > ./configure --host=$HOST --build=`uname -m`-linux-gnu CC="$CC"
> > > AR="$AR" RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP"
> > > CFLAGS="$CFLAGS";
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.synctest/fuego_test.sh
> > > b/engine/tests/Functional.synctest/fuego_test.sh
> > > index aff0820..95d4dda 100755
> > > --- a/engine/tests/Functional.synctest/fuego_test.sh
> > > +++ b/engine/tests/Functional.synctest/fuego_test.sh
> > > @@ -1,7 +1,7 @@
> > > tarball=synctest.tar.gz
> > >
> > > function test_build {
> > > - make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD" &&
> > > touch test_suite_ready || build_error "error while building test"
> > > + make CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" CC="$CC" LD="$LD"
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.thrift/fuego_test.sh
> > > b/engine/tests/Functional.thrift/fuego_test.sh
> > > index 8310e4e..17a8c11 100755
> > > --- a/engine/tests/Functional.thrift/fuego_test.sh
> > > +++ b/engine/tests/Functional.thrift/fuego_test.sh
> > > @@ -111,7 +111,6 @@ function test_build {
> > >
> > > killall CppServer
> > > " > run-tests.sh
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.xorg-macros/fuego_test.sh
> > > b/engine/tests/Functional.xorg-macros/fuego_test.sh
> > > index 95490be..df10fcd 100755
> > > --- a/engine/tests/Functional.xorg-macros/fuego_test.sh
> > > +++ b/engine/tests/Functional.xorg-macros/fuego_test.sh
> > > @@ -6,7 +6,6 @@ function test_build {
> > > if [ -f /usr/share/pkgconfig/xorg-macros.pc ]; then echo 'TEST-2 OK'; else
> > > echo 'TEST-2 FAIL'; fi;
> > > " > run-tests.sh
> > >
> > > - touch test_suite_ready
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/Functional.zlib/fuego_test.sh
> > > b/engine/tests/Functional.zlib/fuego_test.sh
> > > index 7615b5d..c794c52 100755
> > > --- a/engine/tests/Functional.zlib/fuego_test.sh
> > > +++ b/engine/tests/Functional.zlib/fuego_test.sh
> > > @@ -4,7 +4,7 @@ function test_build {
> > > AR=$PREFIX'-ar rc'
> > > CPP=$PREFIX'-gcc -E'
> > > CC="$CC" AR="$AR" CPP="$CPP" ./configure --
> > > includedir=$SDKROOT/usr/include --libdir=$SDKROOT/usr/lib
> > > - make LDSHARED="$CC" >/dev/null && touch test_suite_ready ||
> > > build_error "error while building test"
> > > + make LDSHARED="$CC" >/dev/null
> > > }
> > >
> > > function test_deploy {
> > > diff --git a/engine/tests/OpenSSL/openssl.sh
> > > b/engine/tests/OpenSSL/openssl.sh
> > > index 95848a7..ec062c7 100755
> > > --- a/engine/tests/OpenSSL/openssl.sh
> > > +++ b/engine/tests/OpenSSL/openssl.sh
> > > @@ -51,5 +51,4 @@ function test_build {
> > > ../util/shlib_wrap.sh ./shatest
> > > ../util/shlib_wrap.sh ./ssltest' > run-tests.sh
> > > rm test/fips_aes_data
> > > - touch test_suite_ready || build_error "error while building test"
> > > }
> > > diff --git a/engine/tests/netperf/netperf.sh
> > > b/engine/tests/netperf/netperf.sh
> > > index 2cd6da4..8c14cff 100755
> > > --- a/engine/tests/netperf/netperf.sh
> > > +++ b/engine/tests/netperf/netperf.sh
> > > @@ -1,6 +1,6 @@
> > > function test_build {
> > > echo "ac_cv_func_setpgrp_void=yes" > config.cache
> > > ./configure --build=`./config.guess` --host=$HOST CC="$CC" AR="$AR"
> > > RANLIB="$RANLIB" CXX="$CXX" CPP="$CPP" CXXCPP="$CXXCPP" --config-
> > > cache
> > > - make && touch test_suite_ready || build_error "error while building test"
> > > + make
> > > }
> > >
> > > --
> > > 2.7.4
> > >
> > >
> > > _______________________________________________
> > > Fuego mailing list
> > > Fuego@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
2017-05-16 4:35 ` Daniel Sangorrin
@ 2017-05-16 4:52 ` Bird, Timothy
2017-05-16 5:14 ` Daniel Sangorrin
0 siblings, 1 reply; 31+ messages in thread
From: Bird, Timothy @ 2017-05-16 4:52 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Monday, May 15, 2017 9:35 PM
> To: Bird, Timothy <Tim.Bird@sony.com>; fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test
> developers
>
> Hi Tim,
>
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Tuesday, May 16, 2017 11:35 AM
> > > > function test_build {
> > > > - patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return 1
> > > > + patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return
> > >
> > > Does this end up returning the return value from patch?
> >
> > Yes, return does not alter the $? variable as far as I know.
> >
> > > How does this interact with 'set -e'? I think this should work
> > > since bash doesn't trap command errors when the command
> > > is in a logic expression. But does an error trap from the
> > > non-zero return from test_build?
> >
> > Before this patch, jobs were calling the "build_error" which
> > ended up aborting the job.
> >
> > Now, the return value of test_build is handled in the function "build"
> > using a logic expression (no error trap expected) which calls
> > aboart_job in case of error. So the way it works hasn't really changed.
> >
> > set -e will still trap errors in test_build that do not have a "|| return"
> expression.
>
> I double checked this, an set -e will NOT trap them because the build has a ||
> expression.
> Therefore test writers need to use "|| return" to handle errors within
> test_build.
>
> One alternative is to change
> call_if_present test_build && ret=0 || ret=$?
> for
> call_if_present test_build
> ret=$?
>
> In this case, errors in test_build will be trap and marked as failure by the
> shell.
> Should I change it?
I think so. I think a build error should result in the test being aborted, unless the
test author specifically captures it for some reason.
BTW - I'm not that comfortable with the error handling we have in the code.
set -e has all kinds of gotchas and special cases that IMHO are confusing and
hard to remember for most developers. (see http://mywiki.wooledge.org/BashFAQ/105)
I don't really want test developers to have become shell programming gurus.
However, I do think that that most of the time, we want to stop the test any time
a command or function called by the user fails, and it appears to me that either we
use 'set -e', or we require the user to constantly check return codes in their
own code (in the base script). Maybe we just need to write up some guidelines
for test authors, to let them know the "rules" and some of the gotchas.
We should definitely document somewhere that if the user sees the signal
handler in their console log, something went wrong. Maybe the signal handler
should have better and more prominent error messages to help inform the
user what happened, and maybe we should split the handlers (use different ones
for ERR, INT, etc.) so that we can give more detailed messages about what happened.
What do you think?
-- Tim
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
2017-05-16 4:52 ` Bird, Timothy
@ 2017-05-16 5:14 ` Daniel Sangorrin
0 siblings, 0 replies; 31+ messages in thread
From: Daniel Sangorrin @ 2017-05-16 5:14 UTC (permalink / raw)
To: 'Bird, Timothy', fuego
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> Sent: Tuesday, May 16, 2017 1:53 PM
> To: Daniel Sangorrin; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers
>
>
>
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Monday, May 15, 2017 9:35 PM
> > To: Bird, Timothy <Tim.Bird@sony.com>; fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] [PATCH 09/19] rebuild: make it simpler for test
> > developers
> >
> > Hi Tim,
> >
> > > -----Original Message-----
> > > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > > Sent: Tuesday, May 16, 2017 11:35 AM
> > > > > function test_build {
> > > > > - patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return 1
> > > > > + patch -p0 -N -s < $TEST_HOME/dhry_1.c.patch || return
> > > >
> > > > Does this end up returning the return value from patch?
> > >
> > > Yes, return does not alter the $? variable as far as I know.
> > >
> > > > How does this interact with 'set -e'? I think this should work
> > > > since bash doesn't trap command errors when the command
> > > > is in a logic expression. But does an error trap from the
> > > > non-zero return from test_build?
> > >
> > > Before this patch, jobs were calling the "build_error" which
> > > ended up aborting the job.
> > >
> > > Now, the return value of test_build is handled in the function "build"
> > > using a logic expression (no error trap expected) which calls
> > > aboart_job in case of error. So the way it works hasn't really changed.
> > >
> > > set -e will still trap errors in test_build that do not have a "|| return"
> > expression.
> >
> > I double checked this, an set -e will NOT trap them because the build has a ||
> > expression.
> > Therefore test writers need to use "|| return" to handle errors within
> > test_build.
> >
> > One alternative is to change
> > call_if_present test_build && ret=0 || ret=$?
> > for
> > call_if_present test_build
> > ret=$?
> >
> > In this case, errors in test_build will be trap and marked as failure by the
> > shell.
> > Should I change it?
>
> I think so. I think a build error should result in the test being aborted, unless the
> test author specifically captures it for some reason.
OK, I just sent a patch and pushed it to my next branch.
> BTW - I'm not that comfortable with the error handling we have in the code.
> set -e has all kinds of gotchas and special cases that IMHO are confusing and
> hard to remember for most developers. (see http://mywiki.wooledge.org/BashFAQ/105)
> I don't really want test developers to have become shell programming gurus.
Yes, I agree. I'm not a shell programming guru either, and I don't want to become
one to write code that no one else can read.
> However, I do think that that most of the time, we want to stop the test any time
> a command or function called by the user fails, and it appears to me that either we
> use 'set -e', or we require the user to constantly check return codes in their
> own code (in the base script). Maybe we just need to write up some guidelines
> for test authors, to let them know the "rules" and some of the gotchas.
>
> We should definitely document somewhere that if the user sees the signal
> handler in their console log, something went wrong. Maybe the signal handler
> should have better and more prominent error messages to help inform the
> user what happened, and maybe we should split the handlers (use different ones
> for ERR, INT, etc.) so that we can give more detailed messages about what happened.
>
> What do you think?
In general I think that errors caused by the tests themselves need be handled properly
and written in the final json output.
Errors caused by set -e should normally be BUGs, either in the core code or in the test code, and
happen as rarely as possible. I think that set -e should be only enabled when FUEGO_DEBUG is defined.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2017-05-16 5:14 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-12 8:19 [Fuego] New batch of patches Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 01/19] fail_regexp: rename FAIL_PATTERN to FAIL_REGEXP for coherency Daniel Sangorrin
2017-05-12 22:56 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 02/19] functions:fail_check_cases: add message when no fail cases available Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 03/19] ft2demos: fix test and add to docker Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 04/19] unpack: remove nostrip option Daniel Sangorrin
2017-05-12 22:57 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 05/19] unpack: move code related to spec-defined tarballs to unpack Daniel Sangorrin
2017-05-12 23:05 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 06/19] bc: test that specs can override the tarball variable Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 07/19] unpack: handle possible errors Daniel Sangorrin
2017-05-12 23:09 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 08/19] main: remove ReBuild check Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 09/19] rebuild: make it simpler for test developers Daniel Sangorrin
2017-05-12 23:02 ` Bird, Timothy
2017-05-16 2:35 ` Daniel Sangorrin
2017-05-16 4:35 ` Daniel Sangorrin
2017-05-16 4:52 ` Bird, Timothy
2017-05-16 5:14 ` Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 10/19] call_if_present: tests do not need to implement dummy functions Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 11/19] expat: remove unneeded semicolons Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 12/19] tarball: remove unneeded tarball definitions Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 13/19] style: fix trailing spaces and indentation Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 14/19] parser: allow parser.py to skip processing Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 15/19] LTP: copy target_bin before modifying it Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 16/19] LTP: fix the buildonly and runonly cases Daniel Sangorrin
2017-05-12 8:19 ` [Fuego] [PATCH 17/19] LTP: add support for skipping certain test cases Daniel Sangorrin
2017-05-12 22:57 ` Bird, Timothy
2017-05-12 8:19 ` [Fuego] [PATCH 18/19] LTP: add a spec for docker Daniel Sangorrin
2017-05-12 8:20 ` [Fuego] [PATCH 19/19] LTP: add a fixthis for the -t flag Daniel Sangorrin
2017-05-12 22:52 ` [Fuego] New batch of patches Bird, Timothy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.