* [Fuego] fuego-core patches (migration to testspecs and more)
@ 2017-03-30 1:04 Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
0 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Hi,
This bag of patches includes bug fixes and changes in functionality.
[PATCH 01/16] indentation: fix indentation for dhrystone parser
[PATCH 02/16] abort: fix the abort function
-> without this fix failed jobs were looping until the timeout.
Seems like a Jenkins' bug.
[PATCH 03/16] shell e flag: remove any e flag from fuego
-> As Tim mentioned during the AGL web meeting the e flag is
not the way to catch errors so I removed it.
[PATCH 04/16] parser: remove error message when no matches
[PATCH 05/16] benchmark: use the same pattern as in functional
[PATCH 06/16] board file: remove conf from the path to the board file
-> fuego-ro/boards/ instead of fuego-ro/conf/boards
[PATCH 07/16] batch: remove batch_testplan reference
-> I will implement an add-batch subcommand in the future and
fix reports.sh. For now remove this dead code.
[PATCH 08/16] testplans: use testspecs instead of testplans
-> This is the MOST IMPORTANT patch in this series. It removes
the need to have a testplan or have your test in testplan_default.
Actually testplans are only needed when adding jobs, or
when creating a batch job. The rest can be done with just
the test spec. This also allowed me to simplify ovgen.py
a little bit more.
[PATCH 09/16] distrib: DISTRIB must be defined in the board file
-> I didnt update all of the boards, because I want to completely
rewrite the board files.
[PATCH 10/16] user_checks: add to commands so they execute under
-> thanks for the tip
[PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words
[PATCH 12/16] description setter: put log if the test fails
-> Tim, you were right about this.
[PATCH 13/16] ftc: remove testplans and use testspecs
-> Maybe i should have rebased this and squash it with 08/16 sorry
[PATCH 14/16] ftc test: remove distrib and update paths
[PATCH 15/16] fail_check_cases should not abort the job
-> this might need more review
[PATCH 16/16] a bit of cleaning and style fixes
Thanks,
Daniel
^ permalink raw reply [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser
2017-03-30 1:04 [Fuego] fuego-core patches (migration to testspecs and more) Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 02/16] abort: fix the abort function Daniel Sangorrin
` (14 more replies)
0 siblings, 15 replies; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/tests/Benchmark.Dhrystone/parser.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/engine/tests/Benchmark.Dhrystone/parser.py b/engine/tests/Benchmark.Dhrystone/parser.py
index 58d62ce..d98d2d5 100755
--- a/engine/tests/Benchmark.Dhrystone/parser.py
+++ b/engine/tests/Benchmark.Dhrystone/parser.py
@@ -16,9 +16,9 @@ cur_raw_values = cur_file.readlines()
cur_file.close()
for cur_item in cur_raw_values:
- cur_match = re.match(cur_search_str, cur_item)
- if cur_match:
- cur_dict["Dhrystone"] = cur_match.group(3)
+ cur_match = re.match(cur_search_str, cur_item)
+ if cur_match:
+ cur_dict["Dhrystone"] = cur_match.group(3)
sys.exit(plib.process_data(ref_section_pat, cur_dict, 's', 'FPS'))
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 02/16] abort: fix the abort function
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 20:26 ` Bird, Timothy
2017-03-30 20:29 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 03/16] shell e flag: remove any e flag from fuego Daniel Sangorrin
` (13 subsequent siblings)
14 siblings, 2 replies; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
it seems that BUILD_URL is not being exported, maybe
a bug in Jenkins.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/common.sh | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/engine/scripts/common.sh b/engine/scripts/common.sh
index 8e104cc..3d51fa4 100644
--- a/engine/scripts/common.sh
+++ b/engine/scripts/common.sh
@@ -40,8 +40,11 @@ function abort_job {
echo -e "\n*** ABORTED ***\n"
[ -n "$1" ] && echo -e "Fuego error reason: $1\n"
- wget -qO- ${BUILD_URL}/stop > /dev/null
- while true; do sleep 5; done
+ # TODO: why BUILD_URL is not availables?
+ wget -qO- http://localhost:8080/fuego/job/$JOB_NAME/$BUILD_NUMBER/stop > /dev/null
+ sleep 5
+ echo -e "Jenkins didn't stop the job so let's die by ourselves"
+ false
}
# check is variable is set and fail if otherwise
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 03/16] shell e flag: remove any e flag from fuego
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 02/16] abort: fix the abort function Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 20:40 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 04/16] parser: remove error message when no matches Daniel Sangorrin
` (12 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
This wasn't working well, and we should not need the e flag.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/benchmark.sh | 1 -
engine/scripts/functional.sh | 3 ---
engine/scripts/functions.sh | 7 -------
engine/tests/Functional.LTP/LTP.sh | 2 --
engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh | 3 ---
5 files changed, 16 deletions(-)
diff --git a/engine/scripts/benchmark.sh b/engine/scripts/benchmark.sh
index 1bb67ad..3f9e72c 100644
--- a/engine/scripts/benchmark.sh
+++ b/engine/scripts/benchmark.sh
@@ -24,7 +24,6 @@
if [ -n "$FUEGO_DEBUG" ] ; then
set -x
fi
-set -e
source $FUEGO_CORE/engine/scripts/overlays.sh
set_overlay_vars
diff --git a/engine/scripts/functional.sh b/engine/scripts/functional.sh
index 6a56235..f1e0207 100644
--- a/engine/scripts/functional.sh
+++ b/engine/scripts/functional.sh
@@ -24,7 +24,6 @@
if [ -n "$FUEGO_DEBUG" ] ; then
set -x
fi
-set -e
source $FUEGO_CORE/engine/scripts/overlays.sh
set_overlay_vars
@@ -51,10 +50,8 @@ get_testlog $TESTDIR
echo "##### doing fuego phase: processing ########"
FUEGO_RESULT=0
-set +e
test_processing
export FUEGO_RESULT=$?
-set -e
echo "##### doing fuego phase: post_test ########"
post_test $TESTDIR
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index e9b0f84..b7f0f72 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -422,7 +422,6 @@ function post_test {
dump_syslogs ${fuego_test_tmp} "after"
# Get syslogs
- set +e
get ${fuego_test_tmp}/${NODE_NAME}.${BUILD_ID}.${BUILD_NUMBER}.before ${LOGDIR}/syslog.before.txt
if [ $? -ne 0 ] ; then
echo "Fuego error: Can't read 'before' system log, possibly because /tmp was cleared on boot"
@@ -438,7 +437,6 @@ function post_test {
if [ ! -e $LOGDIR/consolelog.txt ] ; then
ln -s "/var/lib/jenkins/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log" $LOGDIR/consolelog.txt
fi
- set -e
# Remove work and log dirs
[ "$Target_PostCleanup" = "true" ] && target_cleanup $1 || true
@@ -453,14 +451,12 @@ function post_test {
# create functional result file
# don't freak out if the parsing doesn't happen
- set +e
PYTHON_ARGS="-W ignore::DeprecationWarning -W ignore::UserWarning"
if startswith $TESTDIR "Functional." ; then
if [ -e "$TEST_HOME/parser.py" ] ; then
run_python $PYTHON_ARGS "$TEST_HOME/parser.py" -t $TESTDIR -b $NODE_NAME -j $JOB_NAME -n $BUILD_NUMBER -s $BUILD_TIMESTAMP -r $FUEGO_RESULT
fi
fi
- set -e
}
function target_cleanup {
@@ -482,8 +478,6 @@ function target_reboot {
# for target come back up
sleep 10
- # set +e because cmd will fail during the boot process
- set +e
retries=0
while [ $retries -le $1 ]
do
@@ -492,7 +486,6 @@ function target_reboot {
sleep 1
retries=$((retries+1));
done
- set -e
abort_job "FAIL: reboot retries exhausted\n"
}
diff --git a/engine/tests/Functional.LTP/LTP.sh b/engine/tests/Functional.LTP/LTP.sh
index 379626c..1895ed8 100755
--- a/engine/tests/Functional.LTP/LTP.sh
+++ b/engine/tests/Functional.LTP/LTP.sh
@@ -111,9 +111,7 @@ function test_run {
done
# Let some of the tests fail, the information will be in the result xlsx file
- set +e
report "cd $BOARD_TESTDIR/fuego.$TESTDIR; TESTS=\"$TESTS\"; PTSTESTS=\"$PTSTESTS\"; RTTESTS=\"$RTTESTS\"; . ./ltp_target_run.sh"
- set -e
}
function test_processing {
diff --git a/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh b/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
index 8d9dd5a..7c33340 100755
--- a/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
+++ b/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
@@ -37,8 +37,6 @@ function test_processing {
function test_cleanup {
PHASE_STR="${PHASE_STR}:test_cleanup"
- set +e
-
# last chance to put something into the log
# but have to do it directly
LOGFILE=${LOGDIR}/testlog.txt
@@ -71,7 +69,6 @@ function test_cleanup {
rm $TMPFILE2
echo "PHASE_STR=$PHASE_STR" >>$LOGFILE
- set -e
}
. $FUEGO_CORE/engine/scripts/functional.sh
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 04/16] parser: remove error message when no matches
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 02/16] abort: fix the abort function Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 03/16] shell e flag: remove any e flag from fuego Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 20:51 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 05/16] benchmark: use the same pattern as in functional Daniel Sangorrin
` (11 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
This is not supposed to be a parser error, it's rather
an error in the test which is already managed later
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/parser/common.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/engine/scripts/parser/common.py b/engine/scripts/parser/common.py
index d0bdde5..fd74827 100644
--- a/engine/scripts/parser/common.py
+++ b/engine/scripts/parser/common.py
@@ -98,9 +98,8 @@ def write_report_results(rep_data):
def process_data(ref_section_pat, cur_dict, m, label):
- if not cur_dict:
- print "\nFuego error reason: could not parse test results in %s\n" % CUR_LOG
- sys.exit(1)
+ if not cur_dict:
+ sys.exit(1)
if custom_write_report == False:
write_report_results(cur_dict)
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 05/16] benchmark: use the same pattern as in functional
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (2 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 04/16] parser: remove error message when no matches Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 20:54 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 06/16] board file: remove conf from the path to the board file Daniel Sangorrin
` (10 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/benchmark.sh | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/engine/scripts/benchmark.sh b/engine/scripts/benchmark.sh
index 3f9e72c..d666587 100644
--- a/engine/scripts/benchmark.sh
+++ b/engine/scripts/benchmark.sh
@@ -48,12 +48,14 @@ test_run
echo "##### doing fuego phases: get_testlog AND processing ########"
set_testres_file
+
+FUEGO_RESULT=0
bench_processing
export FUEGO_RESULT=$?
-
check_create_logrun
echo "##### doing fuego phase: post_test ########"
post_test $TESTDIR
+echo "Fuego: all test phases complete!"
return $FUEGO_RESULT
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 06/16] board file: remove conf from the path to the board file
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (3 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 05/16] benchmark: use the same pattern as in functional Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 21:24 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 07/16] batch: remove batch_testplan reference Daniel Sangorrin
` (9 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Note: I also simplified the code a bit. Might need review
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc | 9 +++++----
engine/scripts/overlays.sh | 16 ++++------------
2 files changed, 9 insertions(+), 16 deletions(-)
diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index c1aa151..be76427 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -75,7 +75,7 @@ use_statusouput = 1
server = jenkins.Jenkins('http://localhost:8080/fuego')
# keep configuration file in /fuego-ro/conf area
-config_dir = "/fuego-ro/conf"
+config_dir = "/fuego-ro"
CONFIG_FILE = "ftc.conf"
# This is the name of the environment variable
@@ -562,12 +562,13 @@ class run_class:
# return a map of {'<target_name>': target_instance }
def get_fuego_targets(conf):
# scan board directory, and find the board names
- bfile_list = os.listdir(conf.FUEGO_RO + '/conf/boards')
+ # TODO: use glob
+ bfile_list = os.listdir(conf.FUEGO_RO + '/boards')
tmap = {}
for filename in bfile_list:
if filename.endswith(".board"):
- board_path = conf.FUEGO_RO + '/conf/boards/' + filename
+ board_path = conf.FUEGO_RO + '/boards/' + filename
name = filename[:-6]
target = target_class(name, board_path, None, None)
tmap[name] = target
@@ -1789,7 +1790,7 @@ def do_run_test(conf, test_name, target, options):
# FIXTHIS - (!) in run_test, HARDCODE DISTRIB for now (Yuk)
target.env_vars["DISTRIB"]="nosyslogd.dist"
- target.env_vars["BOARD_OVERLAY"]="%s.board" % build_data.target_name
+
for var_name in target.env_vars.keys():
os.environ[var_name] = target.env_vars[var_name]
diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
index a833215..d717e0f 100644
--- a/engine/scripts/overlays.sh
+++ b/engine/scripts/overlays.sh
@@ -38,18 +38,10 @@ OF_TESTPLAN_ARGS=""
OF_DEBUG_ARGS=""
function set_overlay_vars() {
- BOARD_OVERLAY="boards/$NODE_NAME.board"
-
- echo "board overlay: $BOARD_OVERLAY"
-
- if [ "$BOARD_OVERLAY" ] ; then
- echo "using $BOARD_OVERLAY board overlay"
- OF_BOARD_FILE="$FUEGO_RO/conf/$BOARD_OVERLAY"
- if [ ! -f $OF_BOARD_FILE ] ; then
- abort_job "$OF_BOARD_FILE does not exist"
- fi
- else
- abort_job "BOARD_OVERLAY is not defined"
+ # By convention board configuration filenames should be the same as node names
+ OF_BOARD_FILE="$FUEGO_RO/boards/$NODE_NAME.board"
+ if [ ! -f $OF_BOARD_FILE ] ; then
+ abort_job "$OF_BOARD_FILE does not exist"
fi
# check for $DISTRIB and make file path to it
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 07/16] batch: remove batch_testplan reference
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (4 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 06/16] board file: remove conf from the path to the board file Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 21:32 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 08/16] testplans: use testspecs instead of testplans internally Daniel Sangorrin
` (8 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
This is not currently supported. A similar functionality
will need to be provided for reports.sh but probably with
a different interface.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/overlays.sh | 12 +-----------
1 file changed, 1 insertion(+), 11 deletions(-)
diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
index d717e0f..e1d2233 100644
--- a/engine/scripts/overlays.sh
+++ b/engine/scripts/overlays.sh
@@ -55,17 +55,7 @@ function set_overlay_vars() {
abort_job "DISTRIB is not defined"
fi
- # prefer batch testplan over test-specific testplan
- # TODO: fix batch testplans and reports
- if [ "$BATCH_TESTPLAN" ] ; then
- echo "using $BATCH_TESTPLAN batch testplan"
- OF_TESTPLAN="$OF_ROOT/$BATCH_TESTPLAN"
- if [ ! -f $OF_TESTPLAN ] ; then
- abort_job "$OF_TESTPLAN does not exist"
- fi
- OF_TESTPLAN_ARGS="--testplan $OF_TESTPLAN"
-
- elif [ "$TESTPLAN" ] ; then
+ if [ "$TESTPLAN" ] ; then
echo "BATCH_TESTPLAN is not set, using $TESTPLAN.json testplan"
OF_TESTPLAN="$FUEGO_CORE/engine/overlays/testplans/$TESTPLAN.json"
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 08/16] testplans: use testspecs instead of testplans internally
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (5 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 07/16] batch: remove batch_testplan reference Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 21:43 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 09/16] distrib: DISTRIB must be defined in the board file Daniel Sangorrin
` (7 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
This is quite a big change but necessary. Testplans are useful
to batch tests and define which spec or timeout we want for
each test. However, we also want to be able to run tests
individually even if they are not inside a testplan.
Until now one prolog.sh was generated for a testplan. This is not
a good idea for several reasons: collisions between different
test plans or the inability to have the same test with two
different specs in the same testplan.
This patch puts the "prolog.sh" generated each time a job is run
on the corresponding job's log folder. It also modifies the names
of the jobs which are now <board>.<testspec>.<testdir>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 7 +-
engine/scripts/overlays.sh | 36 +++------
engine/scripts/ovgen.py | 191 +++++++++++---------------------------------
3 files changed, 61 insertions(+), 173 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index b7f0f72..2483f61 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -261,9 +261,7 @@ function pre_test {
cmd "true" || abort_job "Cannot connect to $DEVICE via $TRANSPORT"
-# Create the log directory for this test run
export LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAMP}.${BUILD_NUMBER}"
- mkdir -p $LOGDIR
# make a link to the log dir that Jenkins can get to
# (Jenkins might not have the BUILD_TIMESTAMP)
@@ -400,10 +398,11 @@ function post_test {
# reset the signal handler to avoid an infinite loop
trap - SIGTERM SIGHUP SIGALRM SIGINT
+ export LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAMP}.${BUILD_NUMBER}"
+
# source generated prolog.sh file since post_test is called separately
- source $FUEGO_RW/work/${NODE_NAME}_prolog.sh
+ source $LOGDIR/prolog.sh
export SSHPASS=$PASSWORD
- export LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAMP}.${BUILD_NUMBER}"
# re-source params to set correct DEVICE, LOGIN, SSH vars
source $FUEGO_CORE/engine/scripts/params.sh
diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
index e1d2233..cfecc73 100644
--- a/engine/scripts/overlays.sh
+++ b/engine/scripts/overlays.sh
@@ -25,15 +25,13 @@
. $FUEGO_CORE/engine/scripts/common.sh
assert_define "NODE_NAME"
+assert_define "TESTSPEC"
+assert_define "BUILD_NUMBER"
+assert_define "TESTDIR"
-OF_CLASSDIR_ARGS="--classdir $FUEGO_CORE/engine/overlays/base"
-OF_OUTPUT_FILE="$FUEGO_RW/work/${NODE_NAME}_prolog.sh"
-OF_OUTPUT_FILE_ARGS="--output $OF_OUTPUT_FILE"
+OF_CLASSDIRS="$FUEGO_CORE/engine/overlays/base"
OF_OVGEN="$FUEGO_CORE/engine/scripts/ovgen.py"
-OF_DISTRIB_FILE=""
-OF_OVFILES_ARGS=""
-OF_TESTPLAN_ARGS=""
#OF_DEBUG_ARGS="--debug 3"
OF_DEBUG_ARGS=""
@@ -55,29 +53,19 @@ function set_overlay_vars() {
abort_job "DISTRIB is not defined"
fi
- if [ "$TESTPLAN" ] ; then
- echo "BATCH_TESTPLAN is not set, using $TESTPLAN.json testplan"
- OF_TESTPLAN="$FUEGO_CORE/engine/overlays/testplans/$TESTPLAN.json"
+ # Create the log directory for this test run here so we can place the prolog.sh
+ export LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAMP}.${BUILD_NUMBER}"
+ mkdir -p $LOGDIR
- if [ ! -f $OF_TESTPLAN ] ; then
- if [ ! "$(basename $OF_TESTPLAN)" == "testplan_default.json" ] ; then
- abort_job "$OF_TESTPLAN does not exist"
- fi
- fi
- OF_TESTPLAN_ARGS="--testplan $OF_TESTPLAN"
- fi
-
- OF_OVFILES_ARGS="--ovfiles $OF_DISTRIB_FILE $OF_BOARD_FILE"
-
- rm -f $OF_OUTPUT_FILE
+ rm -f $LOGDIR/prolog.sh
- run_python $OF_OVGEN $OF_DEBUG_ARGS $OF_CLASSDIR_ARGS $OF_OVFILES_ARGS $OF_TESTPLAN_ARGS $OF_OUTPUT_FILE_ARGS || abort_job "Error while prolog.sh file generation"
+ run_python $OF_OVGEN $OF_DEBUG_ARGS --classdir $OF_CLASSDIRS --ovfiles $OF_DISTRIB_FILE $OF_BOARD_FILE --testdir $TESTDIR --testspec $TESTSPEC --output $LOGDIR/prolog.sh || abort_job "Error while prolog.sh file generation"
- if [ ! -f "$OF_OUTPUT_FILE" ]
+ if [ ! -f "$LOGDIR/prolog.sh" ]
then
- abort_job "$OF_OUTPUT_FILE not found"
+ abort_job "$LOGDIR/prolog.sh not found"
fi
- source $OF_OUTPUT_FILE
+ source $LOGDIR/prolog.sh
}
diff --git a/engine/scripts/ovgen.py b/engine/scripts/ovgen.py
index 6012453..10d1cf4 100755
--- a/engine/scripts/ovgen.py
+++ b/engine/scripts/ovgen.py
@@ -68,17 +68,15 @@ class OFLayer:
class TestSpecs:
def __init__(self):
- self.testName=""
- self.specList={}
- self.method=None
- self.tarball=None
+ self.name=""
+ self.variables={}
self.fail_case=None
def __repr__(self):
- return "<TestSpecs name:\"%s\" specList:\"%s\" method:%s>\n" % (self.testName, self.specList, self.method)
+ return "<TestSpecs name:\"%s\" variables:\"%s\" fail_case:%s>\n" % (self.name, self.variables, self.fail_case)
def __str__(self):
- return "<TestSpecs name:\"%s\" specList:\"%s\" method:%s>\n" % (self.testName, self.specList, self.method)
+ return "<TestSpecs name:\"%s\" variables:\"%s\" fail_case:%s>\n" % (self.name, self.variables, self.fail_case)
class OFParseException(Exception):
def __init__(self, value):
@@ -377,7 +375,7 @@ def parseOverrideFile(overrideFile, layer, ofcls):
return classes
-def generateProlog(outFilePath, ofcls, classes, tpFiles):
+def generateProlog(outFilePath, ofcls, classes, testdir, testspec):
outfile = open(outFilePath, "w")
for ofc in classes:
@@ -407,43 +405,34 @@ def generateProlog(outFilePath, ofcls, classes, tpFiles):
file.write(outfile, "\n")
- if tpFiles != None:
- for tpf in tpFiles:
- parseGenTestPlan(tpf, outfile, outFilePath)
-
-# generateSpec - generate shell output for all specs of specific test
-# Arguments:
-# curTestSpecs:Dict - selected spec from testplan for current test
-# testName: String - name of test
-# specName: String - name of test spec
-# curSpecs: TestSpecs - all specs for current test
-# fout - file output descriptor opened -> prolog.sh
-
-def generateSpec(curTestSpecs, testName, specName, curSpec, fout):
- del curTestSpecs["name"]
- for par in curTestSpecs:
- if (par == "method" or par == "tarball" or par == "fail_case"):
- continue
+ ts = parseSpec(testdir, testspec)
+ generateSpec(ts, outfile)
+
+# generateSpec - generate shell output for the spec
+# ts:TestSpecs - parsed specs for our testdir
+# testspec: String - name of test spec (e.g. 'default')
+# outFilePath - file output descriptor opened -> prolog.sh
+def generateSpec(ts, fout):
+ debug_print("generating spec '%s' for '%s'" % (ts.variables['name'], ts.name))
- varname = "%s_%s" % (testName, par)
+ for var in ts.variables:
+ if var == 'name':
+ continue
+ varname = "%s_%s" % (ts.name, var)
varname = string.replace(varname, ".", "_").upper()
- value = "%s" % (curTestSpecs[par])
+ value = "%s" % (ts.variables[var])
outStr = '%s="%s"' % (varname, value)
-
- debug_print (outStr, 3)
+ debug_print(outStr, 3)
fout.write(outStr + "\n")
- tNameUp = string.replace(testName, ".", "_").upper()
- tNameUp = string.replace(tNameUp, "-", "_").upper()
-
- #print "cts: %s\n" % (curTestSpecs)
-
- if curSpec.fail_case:
- fc_num = len(curSpec.fail_case)
+ if ts.fail_case:
+ tNameUp = string.replace(ts.name, ".", "_").upper()
+ tNameUp = string.replace(tNameUp, "-", "_").upper()
+ fc_num = len(ts.fail_case)
outNum = "%s_FAIL_CASE_COUNT=%s" % (tNameUp, fc_num)
fout.write(outNum + "\n")
- for fmsg, num in zip(curSpec.fail_case, range(fc_num)):
+ for fmsg, num in zip(ts.fail_case, range(fc_num)):
outPattern = "%s_FAIL_PATTERN_%s=\"%s\"" % (tNameUp, num, fmsg["fail_regexp"])
outMessage = "%s_FAIL_MESSAGE_%s=\"%s\"" % (tNameUp, num, fmsg["fail_message"])
@@ -453,127 +442,46 @@ def generateSpec(curTestSpecs, testName, specName, curSpec, fout):
fout.write(outPattern + "\n")
fout.write(outMessage + "\n")
- if "method" in curTestSpecs:
- if (curTestSpecs["method"] == "tarball"):
- outStr = "%s_METHOD=tarball\n%s_TARBALL_NAME=%s" % (tNameUp, tNameUp, curTestSpecs["tarball"])
- fout.write(outStr + "\n")
- else:
- raise SpecException ("unkown method: %s" % (curTestSpecs["method"]))
- elif curSpec.method:
- if (curSpec.method == "tarball"):
- outStr = ("%s_METHOD=tarball\n%s_TARBALL_NAME=%s" % (tNameUp, tNameUp, curSpec.tarball))
- fout.write(outStr + "\n")
- else:
- raise SpecException ("unkown method: %s" % (curSpec.method))
- else:
- pass
- # TRB 2016-09-09 - ignore missing test spec methods - this feature doesn't work or isn't used
- #print("Can not find method for %s[%s]" % (testName,specName))
- #outStr = ("%s_METHOD=\"none\"" % (tNameUp))
- #fout.write(outStr + "\n")
- # raise SpecException ("Can not find method for %s[%s]" % (testName,specName))
-
-
-def parseGenTestPlan(tpFilePath, fout, fname):
- with open(tpFilePath) as f:
- debug_print("parsing `%s' TP" % (tpFilePath), 1)
- try:
- jd = json.load(f)
- except:
- print "Error parsing testplan file %s" % tpFilePath
- f.seek(0)
- js = json.load(f)
-
- name = jd["testPlanName"]
- fout.write("#testplan: %s\n" % (name))
- for t in jd["tests"]:
- testName = t["testName"]
- specName = t["spec"]
-
- specs = {}
- specs = parseSpecDir("/fuego-core/engine/tests/" + testName)
-
- if testName not in specs:
- raise SpecException("Cannot find test %s in spec list" % (testName))
-
- curSpecs = specs[testName]
-
- if specName not in curSpecs.specList:
- raise SpecException("Cannot find spec %s in %s" % (specName, testName))
-
- curSpecList = curSpecs.specList[specName]
-
- debug_print("generating spec %s for `%s'" % (specName, testName))
- generateSpec(curSpecList, testName, specName, curSpecs, fout)
-
- fout.write("\n")
-
-def parseSpec(specFileName):
+def parseSpec(testdir, testspec):
+ # TODO: get fuego-core from env
+ specpath = '/fuego-core/engine/tests/%s/%s.spec' % (testdir, testdir)
ts = TestSpecs()
- ts.specList={}
- debug_print("Parsing %s spec file" % (specFileName))
+ debug_print("Parsing %s spec file" % (specpath))
- with open(specFileName) as f:
+ with open(specpath) as f:
try:
jd = json.load(f)
except:
- print "Error parsing spec file %s" % specFileName
- f.seek(0)
- jd = json.load(f)
+ raise Exception("Error parsing spec file %s" % specpath)
- name = jd["testName"]
- debug_print("parsing `%s' spec" % (name), 1)
-
- if "method" in jd:
- if jd["method"] == "tarball":
- ts.method = "tarball"
- ts.tarball = jd["tarball"]
- else:
- raise SpecException("%s: Unknown method: %s", (name, ts.tarball))
+ ts.name = jd["testName"]
if "fail_case" in jd:
ts.fail_case = jd["fail_case"]
- debug_print ("Found fail_case msgs for `%s' specs" % (name)
-)
- for spec in jd["specs"]:
- sn = spec["name"]
- ts.testName = name
- ts.specList[sn] = spec
-
- return ts
+ debug_print ("Found fail_case msgs in '%s'" % specpath)
+ for spec in jd["specs"]:
+ if spec["name"] == testspec:
+ ts.variables = spec
-def parseSpecDir(specDir):
- specFiles = glob.glob(specDir + "/*.spec")
- tspList = {}
-
- debug_print ("Found following %s spec files in %s" % (specFiles, specDir))
-
- for sf in specFiles:
- sp = parseSpec(sf)
- tspList[sp.testName] = sp
-
- debug_print ("parsed specs: %s" % (tspList))
-
- return tspList
+ if not ts.variables:
+ raise Exception("Could not find %s in %s" % (testspec, specpath))
+ return ts
def run(test_args=None):
parser = argparse.ArgumentParser(description='Read OF class files, override files and generate prolog defining all variables and functions')
parser.add_argument('--classdir', help='OF base class directory', required=True)
parser.add_argument('--ovfiles', nargs='+', metavar='OVFILE', help='list of directories containing .override files', required=True)
- parser.add_argument('--testplans', nargs='+', metavar='TESTPLAN', help='list of test plan files', required=False)
+ parser.add_argument('--testdir', help='e.g.: Benchmark.Dhrystone', required=True)
+ parser.add_argument('--testspec', nargs='+', metavar='TESTSPEC', help='testspec', required=True)
parser.add_argument('--output', help='output file name', required=True)
parser.add_argument('--debug', help='{1,2,3} debug level (default is no debugging)', type=int, required=False)
args = parser.parse_args(args=test_args)
- classdir = args.classdir
- ovfiles = args.ovfiles
- output = args.output
-
if args.debug:
if args.debug < 1 or args.debug > 3:
@@ -583,29 +491,22 @@ def run(test_args=None):
global log_lvl
log_lvl = args.debug
- testPlans = {}
- if args.testplans != None:
- testPlans = args.testplans
-
- debug_print ("Using =%s, ovfiles=%s" % (classdir, ovfiles), 1)
+ debug_print("Using classdir=%s, ovfiles=%s testdir=%s testspec=%s" % (args.classdir, args.ovfiles, args.testdir, args.testspec), 1)
ofcls = {}
- parseBaseDir(classdir, ofcls)
+ parseBaseDir(args.classdir, ofcls)
layers = {}
classes = []
-
- for ovf in ovfiles:
+ for ovf in args.ovfiles:
classes = classes + parseOverrideFile(ovf, layers, ofcls)
debug_print ("parsed %s override\n------------\n" % (ovf))
- generateProlog(output, ofcls, classes, testPlans)
-
+ generateProlog(args.output, ofcls, classes, args.testdir, args.testspec[0])
def testrun():
- test_args = "--classdir /fuego-core/engine/overlays/base/ --ovfiles /fuego-ro/conf/boards/qemu-arm.board --output prolog.sh --testplans /fuego-core/engine/overlays/testplans/testplan_default.json --debug 2".split()
- # test_args = "--classdir overlays-new/base/ --ovfiles overlays-new/boards/minnow.board --output prolog.sh --debug 2".split()
+ test_args = "--debug 2 --classdir /fuego-core/engine/overlays/base/ --ovfiles /fuego-ro/boards/qemu-arm.board --testdir Benchmark.Dhrystone --testspec default --output prolog.sh".split()
run(test_args)
run()
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 09/16] distrib: DISTRIB must be defined in the board file
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (6 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 08/16] testplans: use testspecs instead of testplans internally Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 22:46 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 10/16] user_checks: add to commands so they execute under jenkins Daniel Sangorrin
` (6 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
If it isn't it defaults to nosyslogd.dist
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc | 21 ++++-----------------
engine/scripts/overlays.sh | 20 +++++++++++---------
2 files changed, 15 insertions(+), 26 deletions(-)
diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index be76427..eb1eb3e 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -741,7 +741,7 @@ def get_includes(include_filename, conf):
inc_vars = parse_shell_file(inc_path, conf)
return inc_vars
-def create_job(board, distrib, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks):
+def create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks):
# flot only necessary for Benchmarks
if str(test).split('.')[0] == 'Benchmark':
flot_link = '<flotile.FlotPublisher plugin="flot@1.0-SNAPSHOT"/>'
@@ -780,7 +780,6 @@ export Target_PreCleanup={precleanup}
export Target_PostCleanup={postcleanup}
export TESTDIR={testdir}
export TESTNAME={testname}
-export DISTRIB="{distrib}"
export TESTPLAN="{testplan}"
timeout --signal=9 {timeout} /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
</command>
@@ -798,7 +797,7 @@ timeout --signal=9 {timeout} /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{
<buildWrappers/>
</project>
""".format(board=board, reboot=str(reboot), rebuild=str(rebuild), precleanup=str(precleanup), postcleanup=str(postcleanup),
- distrib=distrib, testdir=str(test), testname=str(test).split('.')[1],
+ testdir=str(test), testname=str(test).split('.')[1],
testplan=testplan, timeout=timeout, flot_link=flot_link, html_links=html_links))
fd.close()
@@ -929,14 +928,6 @@ def do_add_jobs(conf, options):
else:
raise Exception('No testplan or testcase supplied.')
- if '-d' in options:
- distrib = options[options.index('-d') + 1]
- # TODO: distrib should be in the board's configuration
- options.remove('-d')
- options.remove(distrib)
- else:
- distrib = 'nosyslogd.dist'
-
if test:
timeout = '20m'
reboot = 'false'
@@ -944,11 +935,11 @@ def do_add_jobs(conf, options):
precleanup = 'true'
postcleanup = 'true'
jobextralinks = 'None'
- create_job(board, distrib, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
+ create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
else:
alltests, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks = parse_testplan(testplan)
for test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks in zip(alltests, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks):
- create_job(board, distrib, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
+ create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
create_batch_job(board, testplan, alltests)
sys.exit(0)
@@ -1785,12 +1776,8 @@ def do_run_test(conf, test_name, target, options):
os.environ["NODE_LABELS"] = build_data.target_name
os.environ["Device"] = build_data.target_name
os.environ["TESTDIR"] = build_data.testdir
-
os.environ["WORKSPACE"] = build_data.workspace
- # FIXTHIS - (!) in run_test, HARDCODE DISTRIB for now (Yuk)
- target.env_vars["DISTRIB"]="nosyslogd.dist"
-
for var_name in target.env_vars.keys():
os.environ[var_name] = target.env_vars[var_name]
diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
index cfecc73..2565428 100644
--- a/engine/scripts/overlays.sh
+++ b/engine/scripts/overlays.sh
@@ -42,16 +42,18 @@ function set_overlay_vars() {
abort_job "$OF_BOARD_FILE does not exist"
fi
- # check for $DISTRIB and make file path to it
- if [ "$DISTRIB" ]; then
- echo "using $DISTRIB overlay"
- OF_DISTRIB_FILE="$FUEGO_CORE/engine/overlays/distribs/$DISTRIB"
- if [ ! -f $OF_DISTRIB_FILE ] ; then
- abort_job "$OF_DISTRIB_FILE does not exist"
- fi
- else
- abort_job "DISTRIB is not defined"
+ # check for DISTRIB in the board configuration
+ DISTRIB_LINE=$(grep DISTRIB $OF_BOARD_FILE)
+ eval $DISTRIB_LINE
+ if [ ! "$DISTRIB" ]; then
+ # TODO: automatically discover the best option
+ DISTRIB="nosyslogd.dist"
fi
+ OF_DISTRIB_FILE="$FUEGO_CORE/engine/overlays/distribs/$DISTRIB"
+ if [ ! -f $OF_DISTRIB_FILE ] ; then
+ abort_job "$OF_DISTRIB_FILE does not exist"
+ fi
+ echo "Using $DISTRIB overlay"
# Create the log directory for this test run here so we can place the prolog.sh
export LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAMP}.${BUILD_NUMBER}"
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 10/16] user_checks: add to commands so they execute under jenkins
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (7 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 09/16] distrib: DISTRIB must be defined in the board file Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 22:49 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words Daniel Sangorrin
` (5 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Instead of using sudo -u jenkins ftc xxx use the user_checks
allows to run the command as jenkins even if the caller
is running as root
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index eb1eb3e..e08c56a 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -2769,6 +2769,7 @@ def main():
if command=="add-jobs":
# adds Jenkins jobs
+ user_check()
try:
do_add_jobs(conf, options)
except Exception as e:
@@ -2776,6 +2777,7 @@ def main():
if command=="rm-jobs":
# removes Jenkins jobs
+ user_check()
try:
do_rm_jobs(conf, options)
except Exception as e:
@@ -2783,6 +2785,7 @@ def main():
if command=="add-nodes":
# adds Jenkins nodes
+ user_check()
try:
do_add_nodes(conf, options)
except Exception as e:
@@ -2790,6 +2793,7 @@ def main():
if command=="rm-nodes":
# removes Jenkins nodes
+ user_check()
try:
do_rm_nodes(conf, options)
except Exception as e:
@@ -2801,10 +2805,12 @@ def main():
if command=="list-nodes":
# shows jenkins nodes
+ user_check()
do_list_nodes(conf)
if command=="list-jobs":
# shows jenkins jobs
+ user_check()
do_list_jobs(conf)
if command=="list-plans":
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (8 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 10/16] user_checks: add to commands so they execute under jenkins Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 22:50 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 12/16] description setter: put log if the test fails Daniel Sangorrin
` (4 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
This is so that batch or other jobs can be removed too
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc | 2 --
1 file changed, 2 deletions(-)
diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index e08c56a..6f7d635 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -952,8 +952,6 @@ def do_rm_jobs(conf, options):
else:
for opt in options:
jobs = [job['name'] for job in server.get_jobs()]
- if len(opt.split('.')) != 4:
- raise Exception('%s does not have 4 parts.' % opt)
pattern = '^' + opt
pattern = pattern.replace('.', '\.')
pattern = pattern.replace('*', '\w*')
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 12/16] description setter: put log if the test fails
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (9 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 23:04 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs Daniel Sangorrin
` (3 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Before log was set by default, but as Tim noticed the
fail case was not considered.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index 6f7d635..82a8c18 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -751,6 +751,7 @@ def create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, post
# prepare links for descriptionsetter. Put a link to testlog.txt by default
template_link = '<a href="/fuego/userContent/fuego.logs/%s/${NODE_NAME}.${BUILD_ID}.${BUILD_ID}/%%s">%%s</a>' % test
html_links = template_link % ('testlog.txt', 'log')
+ failed_links = html_links
if jobextralinks != 'None':
for key, value in jobextralinks.iteritems():
html_links = html_links + ' ' + template_link % (value, key)
@@ -788,17 +789,18 @@ timeout --signal=9 {timeout} /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{
<publishers>
{flot_link}
<hudson.plugins.descriptionsetter.DescriptionSetterPublisher plugin="description-setter@1.10">
- <regexp></regexp>
- <regexpForFailed></regexpForFailed>
- <description>{html_links}</description>
- <setForMatrix>false</setForMatrix>
+ <regexp></regexp>
+ <regexpForFailed></regexpForFailed>
+ <description>{html_links}</description>
+ <descriptionForFailed>{failed_links}</descriptionForFailed>
+ <setForMatrix>false</setForMatrix>
</hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
</publishers>
<buildWrappers/>
</project>
""".format(board=board, reboot=str(reboot), rebuild=str(rebuild), precleanup=str(precleanup), postcleanup=str(postcleanup),
testdir=str(test), testname=str(test).split('.')[1],
- testplan=testplan, timeout=timeout, flot_link=flot_link, html_links=html_links))
+ testplan=testplan, timeout=timeout, flot_link=flot_link, html_links=html_links, failed_links=failed_links))
fd.close()
print("Creating job " + test)
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (10 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 12/16] description setter: put log if the test fails Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 23:06 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths Daniel Sangorrin
` (2 subsequent siblings)
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc | 36 +++++++++++++++++++++---------------
1 file changed, 21 insertions(+), 15 deletions(-)
diff --git a/engine/scripts/ftc b/engine/scripts/ftc
index 82a8c18..372c32a 100755
--- a/engine/scripts/ftc
+++ b/engine/scripts/ftc
@@ -85,7 +85,7 @@ TARGET_ENV_VAR="FTC_TARGET"
# format for command_help mapping with: key=name, value=(summary, long description)
command_help = {
"add-jobs":("Adds jobs to Jenkins.",
- """Usage: ftc add-jobs -b <target> [-p <testplan> | -t <testcase>] [-d <foo.dist>]
+ """Usage: ftc add-jobs -b <target> [-p <testplan> | -t <testcase> -s <testspec>]
Example: sudo -u jenkins ftc add-jobs -b docker -p testplan_docker
Example: sudo -u jenkins ftc add-jobs -b docker -t Benchmark.Dhrystone
Use list-plans to see the available test plans.
@@ -93,13 +93,13 @@ command_help = {
This interface may change in the future."""),
"rm-jobs":("Removes jobs from Jenkins.",
- """Usage: ftc rm-jobs <target>.<testplan>.<testcase>
+ """Usage: ftc rm-jobs <target>.<testspec>.<testtype>.<testcase>
Use list-jobs to see the existing jobs. A wildcard can be used to
specify which jobs to remove (just make sure you have 4 words):
Example: sudo -u jenkins ftc rm-jobs docker.testplan_docker.*.*
Example: sudo -u jenkins ftc rm-jobs docker.*.F*.*stress
- Multiple combinations of the <target>.<testplan>.<testcase> pattern
- can be passed as well.
+ Multiple combinations of the <target>.<testspec>.<testtype>.<testcase>
+ pattern can be passed as well.
If no option is provided all existing jobs will be removed."""),
@@ -741,7 +741,7 @@ def get_includes(include_filename, conf):
inc_vars = parse_shell_file(inc_path, conf)
return inc_vars
-def create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks):
+def create_job(board, testspec, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks):
# flot only necessary for Benchmarks
if str(test).split('.')[0] == 'Benchmark':
flot_link = '<flotile.FlotPublisher plugin="flot@1.0-SNAPSHOT"/>'
@@ -781,7 +781,7 @@ export Target_PreCleanup={precleanup}
export Target_PostCleanup={postcleanup}
export TESTDIR={testdir}
export TESTNAME={testname}
-export TESTPLAN="{testplan}"
+export TESTSPEC="{testspec}"
timeout --signal=9 {timeout} /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
</command>
</hudson.tasks.Shell>
@@ -800,13 +800,13 @@ timeout --signal=9 {timeout} /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{
</project>
""".format(board=board, reboot=str(reboot), rebuild=str(rebuild), precleanup=str(precleanup), postcleanup=str(postcleanup),
testdir=str(test), testname=str(test).split('.')[1],
- testplan=testplan, timeout=timeout, flot_link=flot_link, html_links=html_links, failed_links=failed_links))
+ testspec=testspec, timeout=timeout, flot_link=flot_link, html_links=html_links, failed_links=failed_links))
fd.close()
print("Creating job " + test)
try:
subprocess.call('java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego create-job ' +
- board + '.' + testplan + '.' + str(test) + ' < ' + tmp, shell=True)
+ board + '.' + testspec + '.' + str(test) + ' < ' + tmp, shell=True)
subprocess.call('rm -f ' + tmp, shell=True)
except Exception as e:
print("Job already exists")
@@ -862,6 +862,7 @@ def create_batch_job(board, testplan, alltests):
def parse_testplan(testplan):
abspath = '/fuego-core/engine/overlays/testplans/' + testplan + '.json'
alltests = []
+ alltestspecs = []
alltimeouts = []
allextralinks = []
allreboots = []
@@ -873,6 +874,7 @@ def parse_testplan(testplan):
plan = json.load(f)
for test in plan['tests']:
alltests.append(test['testName'])
+ alltestspecs.append(test['spec'])
if 'timeout' in test:
alltimeouts.append(test['timeout'])
else:
@@ -897,7 +899,7 @@ def parse_testplan(testplan):
allextralinks.append(test['extralinks'])
else:
allextralinks.append('None')
- return alltests, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks
+ return alltests, alltestspecs, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks
def do_add_jobs(conf, options):
if '-b' in options:
@@ -925,8 +927,12 @@ def do_add_jobs(conf, options):
raise Exception('Test %s not found.' % test)
options.remove('-t')
options.remove(test)
- # TODO: use spec instead of testplan_default
- testplan = 'testplan_default'
+ if '-s' in options:
+ spec = options[options.index('-s') + 1]
+ options.remove('-s')
+ options.remove(spec)
+ else:
+ spec = 'default'
else:
raise Exception('No testplan or testcase supplied.')
@@ -937,11 +943,11 @@ def do_add_jobs(conf, options):
precleanup = 'true'
postcleanup = 'true'
jobextralinks = 'None'
- create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
+ create_job(board, spec, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
else:
- alltests, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks = parse_testplan(testplan)
- for test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks in zip(alltests, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks):
- create_job(board, testplan, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
+ alltests, alltestspecs, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks = parse_testplan(testplan)
+ for test, spec, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks in zip(alltests, alltestspecs, alltimeouts, allreboots, allrebuilds, allprecleanups, allpostcleanups, allextralinks):
+ create_job(board, spec, test, timeout, reboot, rebuild, precleanup, postcleanup, jobextralinks)
create_batch_job(board, testplan, alltests)
sys.exit(0)
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (11 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 23:13 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 15/16] fail_check_cases should not abort the job Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 16/16] a bit of cleaning and style fixes Daniel Sangorrin
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Note: this is untested and probably won't work but
it's closer to working :)
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/ftc-unit-test.sh | 21 ++-------------------
1 file changed, 2 insertions(+), 19 deletions(-)
diff --git a/engine/scripts/ftc-unit-test.sh b/engine/scripts/ftc-unit-test.sh
index b7d910a..911888e 100755
--- a/engine/scripts/ftc-unit-test.sh
+++ b/engine/scripts/ftc-unit-test.sh
@@ -2,7 +2,7 @@
#
# this script performs a unit test of ftc
#
-TEST_BOARD_FILE=/userdata/conf/boards/ftc-test.board
+TEST_BOARD_FILE=/fuego-ro/boards/ftc-test.board
PATCH_FILE=add-ftc-test-node.patch
#set -x
@@ -100,26 +100,9 @@ cat <<'EOFP' > $PATCH_FILE
+ <mode>NORMAL</mode>
+ <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
+ <launcher class="hudson.slaves.CommandLauncher">
-+ <agentCommand>java -jar /home/jenkins/slave.jar</agentCommand>
++ <agentCommand>java -jar /fuego-core/engine/slave.jar</agentCommand>
+ </launcher>
+ <label></label>
-+ <nodeProperties>
-+ <hudson.slaves.EnvironmentVariablesNodeProperty>
-+ <envVars serialization="custom">
-+ <unserializable-parents/>
-+ <tree-map>
-+ <default>
-+ <comparator class="hudson.util.CaseInsensitiveComparator"/>
-+ </default>
-+ <int>2</int>
-+ <string>BOARD_OVERLAY</string>
-+ <string>boards/ftc-test.board</string>
-+ <string>DISTRIB</string>
-+ <string>distribs/nologger.dist</string>
-+ </tree-map>
-+ </envVars>
-+ </hudson.slaves.EnvironmentVariablesNodeProperty>
-+ </nodeProperties>
+ <userId>anonymous</userId>
+ </slave>
</slaves>
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 15/16] fail_check_cases should not abort the job
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (12 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 23:16 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 16/16] a bit of cleaning and style fixes Daniel Sangorrin
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
failure is something normal and should be kept in a variable
to return it at the end to jenkins. If we just abort then
post_test will not have a chance to return $FUEGO_RESULT
TODO: probably we should change $FUEGO_RESULT though
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index 2483f61..4379271 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -331,7 +331,6 @@ function bench_processing {
}
# search in test log for {!JOB_NAME}_FAIL_PATTERN_n fail cases and abort with message {!JOB_NAME}_FAIL_MESSAGE_n if found
-# args: $1 - path to test log
function fail_check_cases () {
testlog="${LOGDIR}/testlog.txt"
slog_prefix="${LOGDIR}/syslog"
@@ -364,8 +363,7 @@ function fail_check_cases () {
if diff -ua ${slog_prefix}.before ${slog_prefix}.after | grep -vEf "$FUEGO_CORE/engine/scripts/syslog.ignore" | grep -E -e $fptemplate;
then
- echo "Located failing message in syslog diff"
- abort_job "Detected fail message in syslog diff: $fpmessage"
+ echo "Detected fail message in syslog diff: $fpmessage"
else
continue
fi
@@ -373,8 +371,7 @@ function fail_check_cases () {
if grep -e "$fptemplate" $testlog ;
then
- echo "Located failing message in $1"
- abort_job "Detected fail message: $fpmessage"
+ echo "Detected fail message in $testlog: $fpmessage"
fi
done
}
@@ -444,8 +441,10 @@ function post_test {
ov_logger "Test $1 is finished"
# Syslog comparison
+ # TODO: should affect FUEGO_RESULT
syslog_cmp
+ # TODO: should affect FUEGO_RESULT
fail_check_cases || true
# create functional result file
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [Fuego] [PATCH 16/16] a bit of cleaning and style fixes
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
` (13 preceding siblings ...)
2017-03-30 1:04 ` [Fuego] [PATCH 15/16] fail_check_cases should not abort the job Daniel Sangorrin
@ 2017-03-30 1:04 ` Daniel Sangorrin
2017-03-30 23:19 ` Bird, Timothy
14 siblings, 1 reply; 33+ messages in thread
From: Daniel Sangorrin @ 2017-03-30 1:04 UTC (permalink / raw)
To: fuego
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
engine/scripts/functions.sh | 2 +-
engine/scripts/ovgen.py | 17 ++---------------
2 files changed, 3 insertions(+), 16 deletions(-)
diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
index 4379271..e499ab4 100755
--- a/engine/scripts/functions.sh
+++ b/engine/scripts/functions.sh
@@ -500,7 +500,7 @@ function log_compare {
return $FUEGO_RESULT
fi
- cd ${LOGDIR}
+ cd ${LOGDIR}
LOGFILE="testlog.txt"
PARSED_LOGFILE="testlog.${4}.txt"
diff --git a/engine/scripts/ovgen.py b/engine/scripts/ovgen.py
index 10d1cf4..4b25d86 100755
--- a/engine/scripts/ovgen.py
+++ b/engine/scripts/ovgen.py
@@ -37,7 +37,6 @@ OFVAR_DESCRIPTION="DESCRIPTION"
LVAR_NAME="NAME"
LVAR_DESCRIPTION="DESCRIPTION"
-
class OFClass:
def __init__(self):
self.name=""
@@ -65,7 +64,6 @@ class OFLayer:
def __str__(self):
return "<OFLayer name:\"%s\" descr:\"%s\" vars:%s>\n" % (self.name, self.description, self.vars)
-
class TestSpecs:
def __init__(self):
self.name=""
@@ -108,7 +106,6 @@ class SpecException(Exception):
def __str__(self):
return repr(self.value)
-
def debug_print(string, lev=1):
if lev <= log_lvl:
print "log: " + string
@@ -130,7 +127,6 @@ def parseOFVars(line, ofc):
return True
-
# parse variables definitions
def parseVars(line, ofc):
m = re.search("(\w+)=\"(.*)\"", line)
@@ -164,7 +160,6 @@ def parseFunctionBody(name, f):
return funcbody
-
def parseFunction(line, f):
m = re.search("\s?function (\w+).*{", line)
if m==None:
@@ -209,7 +204,6 @@ def parseBaseFile(baseFilePath, ofc):
elif baseParseFunction(line, f, ofc):
debug_print("function found", 3)
-
# parse all base files in dir
def parseBaseDir(baseDirPath, ofcls):
debug_print ("\n------------\nparsing " + baseDirPath + " fuegoclass dir ...\n")
@@ -221,13 +215,12 @@ def parseBaseDir(baseDirPath, ofcls):
debug_print ("parsed %s class\n------------\n" % (ofc.name))
ofcls[ofc.name]=(ofc)
-
def parseInherit(line, ofcls):
m = re.search("inherit \"(.+)\"", line)
- if m==None: return None
+ if m == None:
+ return None
clname = m.group(1)
-
if clname not in ofcls:
raise OFClassNotFoundException("No such class: %s" % (clname))
@@ -259,7 +252,6 @@ def parseLayerVarOverride(line, layer, inhclass):
return True
-
def parseLayerFuncOverride(line, layer, inhclass, f):
m = re.search("override-func (\w+)(.*)", line)
if m==None: return False
@@ -322,7 +314,6 @@ def parseLayerCapList(line, layer, inhclass):
return True
-
def parseOverrideFile(overrideFile, layer, ofcls):
debug_print ("\n-----------\nparsing %s override ...\n" % (overrideFile))
f = open(overrideFile)
@@ -472,7 +463,6 @@ def parseSpec(testdir, testspec):
def run(test_args=None):
parser = argparse.ArgumentParser(description='Read OF class files, override files and generate prolog defining all variables and functions')
-
parser.add_argument('--classdir', help='OF base class directory', required=True)
parser.add_argument('--ovfiles', nargs='+', metavar='OVFILE', help='list of directories containing .override files', required=True)
parser.add_argument('--testdir', help='e.g.: Benchmark.Dhrystone', required=True)
@@ -483,11 +473,9 @@ def run(test_args=None):
args = parser.parse_args(args=test_args)
if args.debug:
-
if args.debug < 1 or args.debug > 3:
print "Error: wrong debug lvl: %s" % (args.debug)
sys.exit
-
global log_lvl
log_lvl = args.debug
@@ -497,7 +485,6 @@ def run(test_args=None):
parseBaseDir(args.classdir, ofcls)
layers = {}
-
classes = []
for ovf in args.ovfiles:
classes = classes + parseOverrideFile(ovf, layers, ofcls)
--
2.7.4
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 02/16] abort: fix the abort function
2017-03-30 1:04 ` [Fuego] [PATCH 02/16] abort: fix the abort function Daniel Sangorrin
@ 2017-03-30 20:26 ` Bird, Timothy
2017-03-30 20:29 ` Bird, Timothy
1 sibling, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 20:26 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> it seems that BUILD_URL is not being exported, maybe
> a bug in Jenkins.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/common.sh | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/engine/scripts/common.sh b/engine/scripts/common.sh
> index 8e104cc..3d51fa4 100644
> --- a/engine/scripts/common.sh
> +++ b/engine/scripts/common.sh
> @@ -40,8 +40,11 @@ function abort_job {
> echo -e "\n*** ABORTED ***\n"
> [ -n "$1" ] && echo -e "Fuego error reason: $1\n"
>
> - wget -qO- ${BUILD_URL}/stop > /dev/null
> - while true; do sleep 5; done
> + # TODO: why BUILD_URL is not availables?
> + wget -qO-
> http://localhost:8080/fuego/job/$JOB_NAME/$BUILD_NUMBER/stop >
> /dev/null
> + sleep 5
Is this enough time? I'd be more comfortable with 30 seconds.
I guess we know at this point that we're not doing any more processing.
I may revisit the timing here when I debug my trap handling for
the script system.
> + echo -e "Jenkins didn't stop the job so let's die by ourselves"
> + false
> }
>
> # check is variable is set and fail if otherwise
> --
> 2.7.4
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 02/16] abort: fix the abort function
2017-03-30 1:04 ` [Fuego] [PATCH 02/16] abort: fix the abort function Daniel Sangorrin
2017-03-30 20:26 ` Bird, Timothy
@ 2017-03-30 20:29 ` Bird, Timothy
1 sibling, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 20:29 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, March 29, 2017 6:05 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 02/16] abort: fix the abort function
>
> it seems that BUILD_URL is not being exported, maybe
> a bug in Jenkins.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/common.sh | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/engine/scripts/common.sh b/engine/scripts/common.sh
> index 8e104cc..3d51fa4 100644
> --- a/engine/scripts/common.sh
> +++ b/engine/scripts/common.sh
> @@ -40,8 +40,11 @@ function abort_job {
> echo -e "\n*** ABORTED ***\n"
> [ -n "$1" ] && echo -e "Fuego error reason: $1\n"
>
> - wget -qO- ${BUILD_URL}/stop > /dev/null
> - while true; do sleep 5; done
> + # TODO: why BUILD_URL is not availables?
One more thing: Can we use FIXTHIS instead of TODO for comments
about things to work on in the future. That's what I've
used in other places and it would be good to be consistent.
I will likely use some code in the fuego wiki to automatically scan
for these types of comments and present them in the wiki.
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 03/16] shell e flag: remove any e flag from fuego
2017-03-30 1:04 ` [Fuego] [PATCH 03/16] shell e flag: remove any e flag from fuego Daniel Sangorrin
@ 2017-03-30 20:40 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 20:40 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> This wasn't working well, and we should not need the e flag.
Hmmm. I'm worried that removing the 'set -e' will lead to error cascades
instead of clean exits in some cases. For example, if something in test_build fails,
do we really want to continue? We haven't added any error handling to compensate
for removing this auto-die functionality.
Right now, the test scripts are written with the assumption that any single
failure should stop the test. I disagree with that assumption, but I feel
like if we change the basic assumption we should be at least looking at the
results from the test_xxx functions (written by the test author), to see
if we should continue or not.
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/benchmark.sh | 1 -
> engine/scripts/functional.sh | 3 ---
> engine/scripts/functions.sh | 7 -------
> engine/tests/Functional.LTP/LTP.sh | 2 --
> engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh | 3 ---
> 5 files changed, 16 deletions(-)
>
> diff --git a/engine/scripts/benchmark.sh b/engine/scripts/benchmark.sh
> index 1bb67ad..3f9e72c 100644
> --- a/engine/scripts/benchmark.sh
> +++ b/engine/scripts/benchmark.sh
> @@ -24,7 +24,6 @@
> if [ -n "$FUEGO_DEBUG" ] ; then
> set -x
> fi
> -set -e
>
> source $FUEGO_CORE/engine/scripts/overlays.sh
> set_overlay_vars
> diff --git a/engine/scripts/functional.sh b/engine/scripts/functional.sh
> index 6a56235..f1e0207 100644
> --- a/engine/scripts/functional.sh
> +++ b/engine/scripts/functional.sh
> @@ -24,7 +24,6 @@
> if [ -n "$FUEGO_DEBUG" ] ; then
> set -x
> fi
> -set -e
>
> source $FUEGO_CORE/engine/scripts/overlays.sh
> set_overlay_vars
> @@ -51,10 +50,8 @@ get_testlog $TESTDIR
>
> echo "##### doing fuego phase: processing ########"
> FUEGO_RESULT=0
> -set +e
> test_processing
> export FUEGO_RESULT=$?
> -set -e
This breaks the logic in test_processing to do short-circuit
execution of multiple log_compare processes. I don't
know if that's a terrible thing or not, but it's a significant
change in program logic.
> echo "##### doing fuego phase: post_test ########"
> post_test $TESTDIR
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index e9b0f84..b7f0f72 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -422,7 +422,6 @@ function post_test {
> dump_syslogs ${fuego_test_tmp} "after"
>
> # Get syslogs
> - set +e
> get
> ${fuego_test_tmp}/${NODE_NAME}.${BUILD_ID}.${BUILD_NUMBER}.befor
> e ${LOGDIR}/syslog.before.txt
> if [ $? -ne 0 ] ; then
> echo "Fuego error: Can't read 'before' system log, possibly because /tmp
> was cleared on boot"
> @@ -438,7 +437,6 @@ function post_test {
> if [ ! -e $LOGDIR/consolelog.txt ] ; then
> ln -s "/var/lib/jenkins/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log"
> $LOGDIR/consolelog.txt
> fi
> - set -e
>
> # Remove work and log dirs
> [ "$Target_PostCleanup" = "true" ] && target_cleanup $1 || true
> @@ -453,14 +451,12 @@ function post_test {
>
> # create functional result file
> # don't freak out if the parsing doesn't happen
> - set +e
> PYTHON_ARGS="-W ignore::DeprecationWarning -W ignore::UserWarning"
> if startswith $TESTDIR "Functional." ; then
> if [ -e "$TEST_HOME/parser.py" ] ; then
> run_python $PYTHON_ARGS "$TEST_HOME/parser.py" -t $TESTDIR -b
> $NODE_NAME -j $JOB_NAME -n $BUILD_NUMBER -s $BUILD_TIMESTAMP -r
> $FUEGO_RESULT
> fi
> fi
> - set -e
> }
>
> function target_cleanup {
> @@ -482,8 +478,6 @@ function target_reboot {
> # for target come back up
> sleep 10
>
> - # set +e because cmd will fail during the boot process
> - set +e
> retries=0
> while [ $retries -le $1 ]
> do
> @@ -492,7 +486,6 @@ function target_reboot {
> sleep 1
> retries=$((retries+1));
> done
> - set -e
> abort_job "FAIL: reboot retries exhausted\n"
> }
>
> diff --git a/engine/tests/Functional.LTP/LTP.sh
> b/engine/tests/Functional.LTP/LTP.sh
> index 379626c..1895ed8 100755
> --- a/engine/tests/Functional.LTP/LTP.sh
> +++ b/engine/tests/Functional.LTP/LTP.sh
> @@ -111,9 +111,7 @@ function test_run {
> done
>
> # Let some of the tests fail, the information will be in the result xlsx file
> - set +e
> report "cd $BOARD_TESTDIR/fuego.$TESTDIR; TESTS=\"$TESTS\";
> PTSTESTS=\"$PTSTESTS\"; RTTESTS=\"$RTTESTS\"; . ./ltp_target_run.sh"
> - set -e
> }
>
> function test_processing {
> diff --git
> a/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
> b/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
> index 8d9dd5a..7c33340 100755
> --- a/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
> +++ b/engine/tests/Functional.fuego_test_phases/fuego_test_phases.sh
> @@ -37,8 +37,6 @@ function test_processing {
> function test_cleanup {
> PHASE_STR="${PHASE_STR}:test_cleanup"
>
> - set +e
> -
> # last chance to put something into the log
> # but have to do it directly
> LOGFILE=${LOGDIR}/testlog.txt
> @@ -71,7 +69,6 @@ function test_cleanup {
> rm $TMPFILE2
>
> echo "PHASE_STR=$PHASE_STR" >>$LOGFILE
> - set -e
> }
>
> . $FUEGO_CORE/engine/scripts/functional.sh
> --
> 2.7.4
I think I'm going to hold off applying this one, until we have more discussion
and possible examination of the effects of it with existing tests.
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 04/16] parser: remove error message when no matches
2017-03-30 1:04 ` [Fuego] [PATCH 04/16] parser: remove error message when no matches Daniel Sangorrin
@ 2017-03-30 20:51 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 20:51 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> This is not supposed to be a parser error, it's rather
> an error in the test which is already managed later
I don't see anywhere else in the parser where failure to find the
benchmark metrics is reported to the user. This test is a bit crude.
A better one would be to check for all metrics that are supposed
to be there. (That test would have to be test-specific, and the
way some parsers work is they don't even have the strings for
the metrics themselves - it comes from the regex applied to the
test log. So maybe this idea is not feasible.)
Is there somewhere else in the system (besides the parser) that catches
this type of error and reports it to the user? (I guess missing benchmark
data might do it.)
-- Tim
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/parser/common.py | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/engine/scripts/parser/common.py
> b/engine/scripts/parser/common.py
> index d0bdde5..fd74827 100644
> --- a/engine/scripts/parser/common.py
> +++ b/engine/scripts/parser/common.py
> @@ -98,9 +98,8 @@ def write_report_results(rep_data):
>
>
> def process_data(ref_section_pat, cur_dict, m, label):
> - if not cur_dict:
> - print "\nFuego error reason: could not parse test results in
> %s\n" % CUR_LOG
> - sys.exit(1)
> + if not cur_dict:
> + sys.exit(1)
>
> if custom_write_report == False:
> write_report_results(cur_dict)
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 05/16] benchmark: use the same pattern as in functional
2017-03-30 1:04 ` [Fuego] [PATCH 05/16] benchmark: use the same pattern as in functional Daniel Sangorrin
@ 2017-03-30 20:54 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 20:54 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/benchmark.sh | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/engine/scripts/benchmark.sh b/engine/scripts/benchmark.sh
> index 3f9e72c..d666587 100644
> --- a/engine/scripts/benchmark.sh
> +++ b/engine/scripts/benchmark.sh
> @@ -48,12 +48,14 @@ test_run
>
> echo "##### doing fuego phases: get_testlog AND processing ########"
> set_testres_file
> +
> +FUEGO_RESULT=0
> bench_processing
> export FUEGO_RESULT=$?
Good catch. One reason I was a bit sloppy here was that I eventually planned
to merge functional.sh and benchmark.sh. They are identical until the last
few steps. But until that merge, these should have the same logic and output.
Thanks.
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 06/16] board file: remove conf from the path to the board file
2017-03-30 1:04 ` [Fuego] [PATCH 06/16] board file: remove conf from the path to the board file Daniel Sangorrin
@ 2017-03-30 21:24 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 21:24 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
> Note: I also simplified the code a bit. Might need review
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc | 9 +++++----
> engine/scripts/overlays.sh | 16 ++++------------
> 2 files changed, 9 insertions(+), 16 deletions(-)
>
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index c1aa151..be76427 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -75,7 +75,7 @@ use_statusouput = 1
> server = jenkins.Jenkins('http://localhost:8080/fuego')
>
> # keep configuration file in /fuego-ro/conf area
> -config_dir = "/fuego-ro/conf"
> +config_dir = "/fuego-ro"
> CONFIG_FILE = "ftc.conf"
This is OK. I think we'll probably have only one configuration
file, so a whole directory is overkill. The configuration file
might migrate to ~/.ftc.conf outside the container.
(Or not - I'm still thinking about this.)
It might end up in /fuego-rw, if I implement something
like 'ftc config' (similar to 'git config') to manage configuration
from the command line. However, disallowing stuff in the
container from overwriting the system config has advantages.
I'm still thinking about this as well.
>
> # This is the name of the environment variable
> @@ -562,12 +562,13 @@ class run_class:
> # return a map of {'<target_name>': target_instance }
> def get_fuego_targets(conf):
> # scan board directory, and find the board names
> - bfile_list = os.listdir(conf.FUEGO_RO + '/conf/boards')
> + # TODO: use glob
> + bfile_list = os.listdir(conf.FUEGO_RO + '/boards')
> tmap = {}
>
> for filename in bfile_list:
> if filename.endswith(".board"):
> - board_path = conf.FUEGO_RO + '/conf/boards/' + filename
> + board_path = conf.FUEGO_RO + '/boards/' + filename
> name = filename[:-6]
> target = target_class(name, board_path, None, None)
> tmap[name] = target
> @@ -1789,7 +1790,7 @@ def do_run_test(conf, test_name, target, options):
>
> # FIXTHIS - (!) in run_test, HARDCODE DISTRIB for now (Yuk)
> target.env_vars["DISTRIB"]="nosyslogd.dist"
> - target.env_vars["BOARD_OVERLAY"]="%s.board" %
> build_data.target_name
> +
This is a nice simplification. Thanks.
> for var_name in target.env_vars.keys():
> os.environ[var_name] = target.env_vars[var_name]
>
> diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
> index a833215..d717e0f 100644
> --- a/engine/scripts/overlays.sh
> +++ b/engine/scripts/overlays.sh
> @@ -38,18 +38,10 @@ OF_TESTPLAN_ARGS=""
> OF_DEBUG_ARGS=""
>
> function set_overlay_vars() {
> - BOARD_OVERLAY="boards/$NODE_NAME.board"
> -
> - echo "board overlay: $BOARD_OVERLAY"
> -
> - if [ "$BOARD_OVERLAY" ] ; then
> - echo "using $BOARD_OVERLAY board overlay"
> - OF_BOARD_FILE="$FUEGO_RO/conf/$BOARD_OVERLAY"
> - if [ ! -f $OF_BOARD_FILE ] ; then
> - abort_job "$OF_BOARD_FILE does not exist"
> - fi
> - else
> - abort_job "BOARD_OVERLAY is not defined"
> + # By convention board configuration filenames should be the same as
> node names
I agree with this. I think using tools to manage the nodes (ftc add-node)
makes this almost foolproof (a user can't mess up the node name), and
it simplifies things nicely. We have a few different rules like this,
that I will start documenting on the wiki. (See http://bird.org/fuego/Fuego_naming_rules)
I may even write a test to validate that a test conforms to the rules.
> + OF_BOARD_FILE="$FUEGO_RO/boards/$NODE_NAME.board"
> + if [ ! -f $OF_BOARD_FILE ] ; then
> + abort_job "$OF_BOARD_FILE does not exist"
> fi
>
> # check for $DISTRIB and make file path to it
> --
> 2.7.4
Looks good.
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 07/16] batch: remove batch_testplan reference
2017-03-30 1:04 ` [Fuego] [PATCH 07/16] batch: remove batch_testplan reference Daniel Sangorrin
@ 2017-03-30 21:32 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 21:32 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> This is not currently supported. A similar functionality
> will need to be provided for reports.sh but probably with
> a different interface.
Agreed.
These used to be set by the "Run SELECTED test on SELECTED targets" job.
If a user has Jenkins with dynamic variable support, this is actually a
fairly nice mechanism.
What happens if you set TESTPLAN in a matrix job in Jenkins?
(Say you had something like: docker.default.batch, but inside
you set the environment variable TESTPLAN=quick (and assuming
the tests in the batch job has 'quick' specs)
Is this the direction you're heading?
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/overlays.sh | 12 +-----------
> 1 file changed, 1 insertion(+), 11 deletions(-)
>
> diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
> index d717e0f..e1d2233 100644
> --- a/engine/scripts/overlays.sh
> +++ b/engine/scripts/overlays.sh
> @@ -55,17 +55,7 @@ function set_overlay_vars() {
> abort_job "DISTRIB is not defined"
> fi
>
> - # prefer batch testplan over test-specific testplan
> - # TODO: fix batch testplans and reports
> - if [ "$BATCH_TESTPLAN" ] ; then
> - echo "using $BATCH_TESTPLAN batch testplan"
> - OF_TESTPLAN="$OF_ROOT/$BATCH_TESTPLAN"
> - if [ ! -f $OF_TESTPLAN ] ; then
> - abort_job "$OF_TESTPLAN does not exist"
> - fi
> - OF_TESTPLAN_ARGS="--testplan $OF_TESTPLAN"
> -
> - elif [ "$TESTPLAN" ] ; then
> + if [ "$TESTPLAN" ] ; then
> echo "BATCH_TESTPLAN is not set, using $TESTPLAN.json testplan"
>
> OF_TESTPLAN="$FUEGO_CORE/engine/overlays/testplans/$TESTPLAN.json"
>
> --
> 2.7.4
Looks good.
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 08/16] testplans: use testspecs instead of testplans internally
2017-03-30 1:04 ` [Fuego] [PATCH 08/16] testplans: use testspecs instead of testplans internally Daniel Sangorrin
@ 2017-03-30 21:43 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 21:43 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> This is quite a big change but necessary. Testplans are useful
> to batch tests and define which spec or timeout we want for
> each test. However, we also want to be able to run tests
> individually even if they are not inside a testplan.
>
> Until now one prolog.sh was generated for a testplan. This is not
> a good idea for several reasons: collisions between different
> test plans or the inability to have the same test with two
> different specs in the same testplan.
>
> This patch puts the "prolog.sh" generated each time a job is run
> on the corresponding job's log folder. It also modifies the names
> of the jobs which are now <board>.<testspec>.<testdir>
Sounds good.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 7 +-
> engine/scripts/overlays.sh | 36 +++------
> engine/scripts/ovgen.py | 191 +++++++++++---------------------------------
> 3 files changed, 61 insertions(+), 173 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index b7f0f72..2483f61 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -261,9 +261,7 @@ function pre_test {
>
> cmd "true" || abort_job "Cannot connect to $DEVICE via $TRANSPORT"
>
> -# Create the log directory for this test run
> export
> LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAM
> P}.${BUILD_NUMBER}"
> - mkdir -p $LOGDIR
>
> # make a link to the log dir that Jenkins can get to
> # (Jenkins might not have the BUILD_TIMESTAMP)
> @@ -400,10 +398,11 @@ function post_test {
> # reset the signal handler to avoid an infinite loop
> trap - SIGTERM SIGHUP SIGALRM SIGINT
>
> + export
> LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAM
> P}.${BUILD_NUMBER}"
> +
> # source generated prolog.sh file since post_test is called separately
> - source $FUEGO_RW/work/${NODE_NAME}_prolog.sh
> + source $LOGDIR/prolog.sh
> export SSHPASS=$PASSWORD
> - export
> LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAM
> P}.${BUILD_NUMBER}"
>
> # re-source params to set correct DEVICE, LOGIN, SSH vars
> source $FUEGO_CORE/engine/scripts/params.sh
> diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
> index e1d2233..cfecc73 100644
> --- a/engine/scripts/overlays.sh
> +++ b/engine/scripts/overlays.sh
> @@ -25,15 +25,13 @@
> . $FUEGO_CORE/engine/scripts/common.sh
>
> assert_define "NODE_NAME"
> +assert_define "TESTSPEC"
> +assert_define "BUILD_NUMBER"
> +assert_define "TESTDIR"
>
> -OF_CLASSDIR_ARGS="--classdir $FUEGO_CORE/engine/overlays/base"
> -OF_OUTPUT_FILE="$FUEGO_RW/work/${NODE_NAME}_prolog.sh"
> -OF_OUTPUT_FILE_ARGS="--output $OF_OUTPUT_FILE"
> +OF_CLASSDIRS="$FUEGO_CORE/engine/overlays/base"
> OF_OVGEN="$FUEGO_CORE/engine/scripts/ovgen.py"
>
> -OF_DISTRIB_FILE=""
> -OF_OVFILES_ARGS=""
> -OF_TESTPLAN_ARGS=""
> #OF_DEBUG_ARGS="--debug 3"
> OF_DEBUG_ARGS=""
>
> @@ -55,29 +53,19 @@ function set_overlay_vars() {
> abort_job "DISTRIB is not defined"
> fi
>
> - if [ "$TESTPLAN" ] ; then
> - echo "BATCH_TESTPLAN is not set, using $TESTPLAN.json testplan"
> -
> OF_TESTPLAN="$FUEGO_CORE/engine/overlays/testplans/$TESTPLAN.json"
> + # Create the log directory for this test run here so we can place the
> prolog.sh
> + export
> LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAM
> P}.${BUILD_NUMBER}"
> + mkdir -p $LOGDIR
>
> - if [ ! -f $OF_TESTPLAN ] ; then
> - if [ ! "$(basename $OF_TESTPLAN)" == "testplan_default.json" ] ; then
> - abort_job "$OF_TESTPLAN does not exist"
> - fi
> - fi
> - OF_TESTPLAN_ARGS="--testplan $OF_TESTPLAN"
> - fi
> -
> - OF_OVFILES_ARGS="--ovfiles $OF_DISTRIB_FILE $OF_BOARD_FILE"
> -
> - rm -f $OF_OUTPUT_FILE
> + rm -f $LOGDIR/prolog.sh
>
> - run_python $OF_OVGEN $OF_DEBUG_ARGS $OF_CLASSDIR_ARGS
> $OF_OVFILES_ARGS $OF_TESTPLAN_ARGS $OF_OUTPUT_FILE_ARGS ||
> abort_job "Error while prolog.sh file generation"
> + run_python $OF_OVGEN $OF_DEBUG_ARGS --classdir $OF_CLASSDIRS --
> ovfiles $OF_DISTRIB_FILE $OF_BOARD_FILE --testdir $TESTDIR --testspec
> $TESTSPEC --output $LOGDIR/prolog.sh || abort_job "Error while prolog.sh
> file generation"
>
> - if [ ! -f "$OF_OUTPUT_FILE" ]
> + if [ ! -f "$LOGDIR/prolog.sh" ]
> then
> - abort_job "$OF_OUTPUT_FILE not found"
> + abort_job "$LOGDIR/prolog.sh not found"
> fi
>
> - source $OF_OUTPUT_FILE
> + source $LOGDIR/prolog.sh
> }
>
> diff --git a/engine/scripts/ovgen.py b/engine/scripts/ovgen.py
> index 6012453..10d1cf4 100755
> --- a/engine/scripts/ovgen.py
> +++ b/engine/scripts/ovgen.py
> @@ -68,17 +68,15 @@ class OFLayer:
>
> class TestSpecs:
> def __init__(self):
> - self.testName=""
> - self.specList={}
> - self.method=None
> - self.tarball=None
> + self.name=""
> + self.variables={}
> self.fail_case=None
>
> def __repr__(self):
> - return "<TestSpecs name:\"%s\" specList:\"%s\" method:%s>\n" %
> (self.testName, self.specList, self.method)
> + return "<TestSpecs name:\"%s\" variables:\"%s\" fail_case:%s>\n" %
> (self.name, self.variables, self.fail_case)
>
> def __str__(self):
> - return "<TestSpecs name:\"%s\" specList:\"%s\" method:%s>\n" %
> (self.testName, self.specList, self.method)
> + return "<TestSpecs name:\"%s\" variables:\"%s\" fail_case:%s>\n" %
> (self.name, self.variables, self.fail_case)
>
> class OFParseException(Exception):
> def __init__(self, value):
> @@ -377,7 +375,7 @@ def parseOverrideFile(overrideFile, layer, ofcls):
>
> return classes
>
> -def generateProlog(outFilePath, ofcls, classes, tpFiles):
> +def generateProlog(outFilePath, ofcls, classes, testdir, testspec):
> outfile = open(outFilePath, "w")
>
> for ofc in classes:
> @@ -407,43 +405,34 @@ def generateProlog(outFilePath, ofcls, classes,
> tpFiles):
>
> file.write(outfile, "\n")
>
> - if tpFiles != None:
> - for tpf in tpFiles:
> - parseGenTestPlan(tpf, outfile, outFilePath)
> -
> -# generateSpec - generate shell output for all specs of specific test
> -# Arguments:
> -# curTestSpecs:Dict - selected spec from testplan for current test
> -# testName: String - name of test
> -# specName: String - name of test spec
> -# curSpecs: TestSpecs - all specs for current test
> -# fout - file output descriptor opened -> prolog.sh
> -
> -def generateSpec(curTestSpecs, testName, specName, curSpec, fout):
> - del curTestSpecs["name"]
> - for par in curTestSpecs:
> - if (par == "method" or par == "tarball" or par == "fail_case"):
> - continue
> + ts = parseSpec(testdir, testspec)
> + generateSpec(ts, outfile)
> +
> +# generateSpec - generate shell output for the spec
> +# ts:TestSpecs - parsed specs for our testdir
> +# testspec: String - name of test spec (e.g. 'default')
> +# outFilePath - file output descriptor opened -> prolog.sh
> +def generateSpec(ts, fout):
> + debug_print("generating spec '%s' for '%s'" % (ts.variables['name'],
> ts.name))
>
> - varname = "%s_%s" % (testName, par)
> + for var in ts.variables:
> + if var == 'name':
> + continue
> + varname = "%s_%s" % (ts.name, var)
> varname = string.replace(varname, ".", "_").upper()
> - value = "%s" % (curTestSpecs[par])
> + value = "%s" % (ts.variables[var])
> outStr = '%s="%s"' % (varname, value)
> -
> - debug_print (outStr, 3)
> + debug_print(outStr, 3)
> fout.write(outStr + "\n")
>
> - tNameUp = string.replace(testName, ".", "_").upper()
> - tNameUp = string.replace(tNameUp, "-", "_").upper()
> -
> - #print "cts: %s\n" % (curTestSpecs)
> -
> - if curSpec.fail_case:
> - fc_num = len(curSpec.fail_case)
> + if ts.fail_case:
> + tNameUp = string.replace(ts.name, ".", "_").upper()
> + tNameUp = string.replace(tNameUp, "-", "_").upper()
> + fc_num = len(ts.fail_case)
> outNum = "%s_FAIL_CASE_COUNT=%s" % (tNameUp, fc_num)
> fout.write(outNum + "\n")
>
> - for fmsg, num in zip(curSpec.fail_case, range(fc_num)):
> + for fmsg, num in zip(ts.fail_case, range(fc_num)):
> outPattern = "%s_FAIL_PATTERN_%s=\"%s\"" % (tNameUp, num,
> fmsg["fail_regexp"])
> outMessage = "%s_FAIL_MESSAGE_%s=\"%s\"" % (tNameUp, num,
> fmsg["fail_message"])
>
> @@ -453,127 +442,46 @@ def generateSpec(curTestSpecs, testName,
> specName, curSpec, fout):
> fout.write(outPattern + "\n")
> fout.write(outMessage + "\n")
>
> - if "method" in curTestSpecs:
> - if (curTestSpecs["method"] == "tarball"):
> - outStr = "%s_METHOD=tarball\n%s_TARBALL_NAME=%s" %
> (tNameUp, tNameUp, curTestSpecs["tarball"])
> - fout.write(outStr + "\n")
> - else:
> - raise SpecException ("unkown method: %s" %
> (curTestSpecs["method"]))
> - elif curSpec.method:
> - if (curSpec.method == "tarball"):
> - outStr = ("%s_METHOD=tarball\n%s_TARBALL_NAME=%s" %
> (tNameUp, tNameUp, curSpec.tarball))
> - fout.write(outStr + "\n")
> - else:
> - raise SpecException ("unkown method: %s" % (curSpec.method))
> - else:
> - pass
> - # TRB 2016-09-09 - ignore missing test spec methods - this feature
> doesn't work or isn't used
> - #print("Can not find method for %s[%s]" % (testName,specName))
> - #outStr = ("%s_METHOD=\"none\"" % (tNameUp))
> - #fout.write(outStr + "\n")
> - # raise SpecException ("Can not find method for %s[%s]" %
> (testName,specName))
> -
> -
> -def parseGenTestPlan(tpFilePath, fout, fname):
> - with open(tpFilePath) as f:
> - debug_print("parsing `%s' TP" % (tpFilePath), 1)
> - try:
> - jd = json.load(f)
> - except:
> - print "Error parsing testplan file %s" % tpFilePath
> - f.seek(0)
> - js = json.load(f)
> -
> - name = jd["testPlanName"]
> - fout.write("#testplan: %s\n" % (name))
> - for t in jd["tests"]:
> - testName = t["testName"]
> - specName = t["spec"]
> -
> - specs = {}
> - specs = parseSpecDir("/fuego-core/engine/tests/" + testName)
> -
> - if testName not in specs:
> - raise SpecException("Cannot find test %s in spec list" % (testName))
> -
> - curSpecs = specs[testName]
> -
> - if specName not in curSpecs.specList:
> - raise SpecException("Cannot find spec %s in %s" % (specName,
> testName))
> -
> - curSpecList = curSpecs.specList[specName]
> -
> - debug_print("generating spec %s for `%s'" % (specName, testName))
> - generateSpec(curSpecList, testName, specName, curSpecs, fout)
> -
> - fout.write("\n")
> -
> -def parseSpec(specFileName):
> +def parseSpec(testdir, testspec):
> + # TODO: get fuego-core from env
> + specpath = '/fuego-core/engine/tests/%s/%s.spec' % (testdir, testdir)
> ts = TestSpecs()
> - ts.specList={}
>
> - debug_print("Parsing %s spec file" % (specFileName))
> + debug_print("Parsing %s spec file" % (specpath))
>
> - with open(specFileName) as f:
> + with open(specpath) as f:
> try:
> jd = json.load(f)
> except:
> - print "Error parsing spec file %s" % specFileName
> - f.seek(0)
> - jd = json.load(f)
> + raise Exception("Error parsing spec file %s" % specpath)
>
> - name = jd["testName"]
> - debug_print("parsing `%s' spec" % (name), 1)
> -
> - if "method" in jd:
> - if jd["method"] == "tarball":
> - ts.method = "tarball"
> - ts.tarball = jd["tarball"]
> - else:
> - raise SpecException("%s: Unknown method: %s", (name, ts.tarball))
> + ts.name = jd["testName"]
>
> if "fail_case" in jd:
> ts.fail_case = jd["fail_case"]
> - debug_print ("Found fail_case msgs for `%s' specs" % (name)
> -)
> - for spec in jd["specs"]:
> - sn = spec["name"]
> - ts.testName = name
> - ts.specList[sn] = spec
> -
> - return ts
> + debug_print ("Found fail_case msgs in '%s'" % specpath)
>
> + for spec in jd["specs"]:
> + if spec["name"] == testspec:
> + ts.variables = spec
>
> -def parseSpecDir(specDir):
> - specFiles = glob.glob(specDir + "/*.spec")
> - tspList = {}
> -
> - debug_print ("Found following %s spec files in %s" % (specFiles, specDir))
> -
> - for sf in specFiles:
> - sp = parseSpec(sf)
> - tspList[sp.testName] = sp
> -
> - debug_print ("parsed specs: %s" % (tspList))
> -
> - return tspList
> + if not ts.variables:
> + raise Exception("Could not find %s in %s" % (testspec, specpath))
>
> + return ts
>
> def run(test_args=None):
> parser = argparse.ArgumentParser(description='Read OF class files,
> override files and generate prolog defining all variables and functions')
>
> parser.add_argument('--classdir', help='OF base class directory',
> required=True)
> parser.add_argument('--ovfiles', nargs='+', metavar='OVFILE', help='list of
> directories containing .override files', required=True)
> - parser.add_argument('--testplans', nargs='+', metavar='TESTPLAN',
> help='list of test plan files', required=False)
> + parser.add_argument('--testdir', help='e.g.: Benchmark.Dhrystone',
> required=True)
> + parser.add_argument('--testspec', nargs='+', metavar='TESTSPEC',
> help='testspec', required=True)
> parser.add_argument('--output', help='output file name', required=True)
> parser.add_argument('--debug', help='{1,2,3} debug level (default is no
> debugging)', type=int, required=False)
>
> args = parser.parse_args(args=test_args)
>
> - classdir = args.classdir
> - ovfiles = args.ovfiles
> - output = args.output
> -
> if args.debug:
>
> if args.debug < 1 or args.debug > 3:
> @@ -583,29 +491,22 @@ def run(test_args=None):
> global log_lvl
> log_lvl = args.debug
>
> - testPlans = {}
> - if args.testplans != None:
> - testPlans = args.testplans
> -
> - debug_print ("Using =%s, ovfiles=%s" % (classdir, ovfiles), 1)
> + debug_print("Using classdir=%s, ovfiles=%s testdir=%s testspec=%s" %
> (args.classdir, args.ovfiles, args.testdir, args.testspec), 1)
>
> ofcls = {}
> - parseBaseDir(classdir, ofcls)
> + parseBaseDir(args.classdir, ofcls)
>
> layers = {}
>
> classes = []
> -
> - for ovf in ovfiles:
> + for ovf in args.ovfiles:
> classes = classes + parseOverrideFile(ovf, layers, ofcls)
> debug_print ("parsed %s override\n------------\n" % (ovf))
>
> - generateProlog(output, ofcls, classes, testPlans)
> -
> + generateProlog(args.output, ofcls, classes, args.testdir, args.testspec[0])
>
> def testrun():
> - test_args = "--classdir /fuego-core/engine/overlays/base/ --ovfiles
> /fuego-ro/conf/boards/qemu-arm.board --output prolog.sh --testplans
> /fuego-core/engine/overlays/testplans/testplan_default.json --debug
> 2".split()
> - # test_args = "--classdir overlays-new/base/ --ovfiles overlays-
> new/boards/minnow.board --output prolog.sh --debug 2".split()
> + test_args = "--debug 2 --classdir /fuego-core/engine/overlays/base/ --
> ovfiles /fuego-ro/boards/qemu-arm.board --testdir Benchmark.Dhrystone --
> testspec default --output prolog.sh".split()
> run(test_args)
>
> run()
> --
OK - good stuff.
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 09/16] distrib: DISTRIB must be defined in the board file
2017-03-30 1:04 ` [Fuego] [PATCH 09/16] distrib: DISTRIB must be defined in the board file Daniel Sangorrin
@ 2017-03-30 22:46 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 22:46 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> If it isn't it defaults to nosyslogd.dist
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc | 21 ++++-----------------
> engine/scripts/overlays.sh | 20 +++++++++++---------
> 2 files changed, 15 insertions(+), 26 deletions(-)
>
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index be76427..eb1eb3e 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -741,7 +741,7 @@ def get_includes(include_filename, conf):
> inc_vars = parse_shell_file(inc_path, conf)
> return inc_vars
>
> -def create_job(board, distrib, testplan, test, timeout, reboot, rebuild,
> precleanup, postcleanup, jobextralinks):
> +def create_job(board, testplan, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks):
> # flot only necessary for Benchmarks
> if str(test).split('.')[0] == 'Benchmark':
> flot_link = '<flotile.FlotPublisher plugin="flot@1.0-SNAPSHOT"/>'
> @@ -780,7 +780,6 @@ export Target_PreCleanup={precleanup}
> export Target_PostCleanup={postcleanup}
> export TESTDIR={testdir}
> export TESTNAME={testname}
> -export DISTRIB="{distrib}"
> export TESTPLAN="{testplan}"
> timeout --signal=9 {timeout} /bin/bash
> $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
> </command>
> @@ -798,7 +797,7 @@ timeout --signal=9 {timeout} /bin/bash
> $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{
> <buildWrappers/>
> </project>
> """.format(board=board, reboot=str(reboot), rebuild=str(rebuild),
> precleanup=str(precleanup), postcleanup=str(postcleanup),
> - distrib=distrib, testdir=str(test), testname=str(test).split('.')[1],
> + testdir=str(test), testname=str(test).split('.')[1],
> testplan=testplan, timeout=timeout, flot_link=flot_link,
> html_links=html_links))
> fd.close()
>
> @@ -929,14 +928,6 @@ def do_add_jobs(conf, options):
> else:
> raise Exception('No testplan or testcase supplied.')
>
> - if '-d' in options:
> - distrib = options[options.index('-d') + 1]
> - # TODO: distrib should be in the board's configuration
> - options.remove('-d')
> - options.remove(distrib)
> - else:
> - distrib = 'nosyslogd.dist'
> -
> if test:
> timeout = '20m'
> reboot = 'false'
> @@ -944,11 +935,11 @@ def do_add_jobs(conf, options):
> precleanup = 'true'
> postcleanup = 'true'
> jobextralinks = 'None'
> - create_job(board, distrib, testplan, test, timeout, reboot, rebuild,
> precleanup, postcleanup, jobextralinks)
> + create_job(board, testplan, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks)
> else:
> alltests, alltimeouts, allreboots, allrebuilds, allprecleanups,
> allpostcleanups, allextralinks = parse_testplan(testplan)
> for test, timeout, reboot, rebuild, precleanup, postcleanup,
> jobextralinks in zip(alltests, alltimeouts, allreboots, allrebuilds,
> allprecleanups, allpostcleanups, allextralinks):
> - create_job(board, distrib, testplan, test, timeout, reboot, rebuild,
> precleanup, postcleanup, jobextralinks)
> + create_job(board, testplan, test, timeout, reboot, rebuild,
> precleanup, postcleanup, jobextralinks)
> create_batch_job(board, testplan, alltests)
> sys.exit(0)
>
> @@ -1785,12 +1776,8 @@ def do_run_test(conf, test_name, target,
> options):
> os.environ["NODE_LABELS"] = build_data.target_name
> os.environ["Device"] = build_data.target_name
> os.environ["TESTDIR"] = build_data.testdir
> -
> os.environ["WORKSPACE"] = build_data.workspace
>
> - # FIXTHIS - (!) in run_test, HARDCODE DISTRIB for now (Yuk)
> - target.env_vars["DISTRIB"]="nosyslogd.dist"
>
Hey! Thanks for cleaning up my code!
-
> for var_name in target.env_vars.keys():
> os.environ[var_name] = target.env_vars[var_name]
>
> diff --git a/engine/scripts/overlays.sh b/engine/scripts/overlays.sh
> index cfecc73..2565428 100644
> --- a/engine/scripts/overlays.sh
> +++ b/engine/scripts/overlays.sh
> @@ -42,16 +42,18 @@ function set_overlay_vars() {
> abort_job "$OF_BOARD_FILE does not exist"
> fi
>
> - # check for $DISTRIB and make file path to it
> - if [ "$DISTRIB" ]; then
> - echo "using $DISTRIB overlay"
> - OF_DISTRIB_FILE="$FUEGO_CORE/engine/overlays/distribs/$DISTRIB"
> - if [ ! -f $OF_DISTRIB_FILE ] ; then
> - abort_job "$OF_DISTRIB_FILE does not exist"
> - fi
> - else
> - abort_job "DISTRIB is not defined"
> + # check for DISTRIB in the board configuration
> + DISTRIB_LINE=$(grep DISTRIB $OF_BOARD_FILE)
> + eval $DISTRIB_LINE
Ok - that's a bit dicey. Hopefully no one ever puts:
"rm -rf / ; DISTRIB=foo" in their board file.
I think we should parse this instead of evaluate it.
I modified the patch to do this.
> + if [ ! "$DISTRIB" ]; then
> + # TODO: automatically discover the best option
> + DISTRIB="nosyslogd.dist"
> fi
> + OF_DISTRIB_FILE="$FUEGO_CORE/engine/overlays/distribs/$DISTRIB"
> + if [ ! -f $OF_DISTRIB_FILE ] ; then
> + abort_job "$OF_DISTRIB_FILE does not exist"
> + fi
> + echo "Using $DISTRIB overlay"
>
> # Create the log directory for this test run here so we can place the
> prolog.sh
> export
> LOGDIR="$FUEGO_RW/logs/$TESTDIR/${NODE_NAME}.${BUILD_TIMESTAM
> P}.${BUILD_NUMBER}"
> --
> 2.7.4
Acked-by: Tim Bird
Thanks.
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 10/16] user_checks: add to commands so they execute under jenkins
2017-03-30 1:04 ` [Fuego] [PATCH 10/16] user_checks: add to commands so they execute under jenkins Daniel Sangorrin
@ 2017-03-30 22:49 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 22:49 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> Instead of using sudo -u jenkins ftc xxx use the user_checks
> allows to run the command as jenkins even if the caller
> is running as root
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index eb1eb3e..e08c56a 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -2769,6 +2769,7 @@ def main():
>
> if command=="add-jobs":
> # adds Jenkins jobs
> + user_check()
> try:
> do_add_jobs(conf, options)
> except Exception as e:
> @@ -2776,6 +2777,7 @@ def main():
>
> if command=="rm-jobs":
> # removes Jenkins jobs
> + user_check()
> try:
> do_rm_jobs(conf, options)
> except Exception as e:
> @@ -2783,6 +2785,7 @@ def main():
>
> if command=="add-nodes":
> # adds Jenkins nodes
> + user_check()
> try:
> do_add_nodes(conf, options)
> except Exception as e:
> @@ -2790,6 +2793,7 @@ def main():
>
> if command=="rm-nodes":
> # removes Jenkins nodes
> + user_check()
> try:
> do_rm_nodes(conf, options)
> except Exception as e:
> @@ -2801,10 +2805,12 @@ def main():
>
> if command=="list-nodes":
> # shows jenkins nodes
> + user_check()
> do_list_nodes(conf)
>
> if command=="list-jobs":
> # shows jenkins jobs
> + user_check()
> do_list_jobs(conf)
>
> if command=="list-plans":
> --
> 2.7.4
Looks good. I'm still wondering why these need to be run as 'jenkins'.
I have run them as root without problems. Is this a permission thing
with Jenkins security configured a certain way?
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words
2017-03-30 1:04 ` [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words Daniel Sangorrin
@ 2017-03-30 22:50 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 22:50 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words
>
> This is so that batch or other jobs can be removed too
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index e08c56a..6f7d635 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -952,8 +952,6 @@ def do_rm_jobs(conf, options):
> else:
> for opt in options:
> jobs = [job['name'] for job in server.get_jobs()]
> - if len(opt.split('.')) != 4:
> - raise Exception('%s does not have 4 parts.' % opt)
> pattern = '^' + opt
> pattern = pattern.replace('.', '\.')
> pattern = pattern.replace('*', '\w*')
> --
> 2.7.4
Acked-by: Tim Bird
-- Tim
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 12/16] description setter: put log if the test fails
2017-03-30 1:04 ` [Fuego] [PATCH 12/16] description setter: put log if the test fails Daniel Sangorrin
@ 2017-03-30 23:04 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 23:04 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
Looks good.
Acked-by: Tim Bird
-- Tim
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, March 29, 2017 6:05 PM
>
> Before log was set by default, but as Tim noticed the
> fail case was not considered.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc | 12 +++++++-----
> 1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index 6f7d635..82a8c18 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -751,6 +751,7 @@ def create_job(board, testplan, test, timeout, reboot,
> rebuild, precleanup, post
> # prepare links for descriptionsetter. Put a link to testlog.txt by default
> template_link = '<a
> href="/fuego/userContent/fuego.logs/%s/${NODE_NAME}.${BUILD_I
> D}.${BUILD_ID}/%%s">%%s</a>' % test
> html_links = template_link % ('testlog.txt', 'log')
> + failed_links = html_links
> if jobextralinks != 'None':
> for key, value in jobextralinks.iteritems():
> html_links = html_links + ' ' + template_link % (value, key)
> @@ -788,17 +789,18 @@ timeout --signal=9 {timeout} /bin/bash
> $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{
> <publishers>
> {flot_link}
> <hudson.plugins.descriptionsetter.DescriptionSetterPublisher
> plugin="description-setter@1.10">
> - <regexp></regexp>
> - <regexpForFailed></regexpForFailed>
> - <description>{html_links}</description>
> - <setForMatrix>false</setForMatrix>
> + <regexp></regexp>
> + <regexpForFailed></regexpForFailed>
> + <description>{html_links}</description>
> + <descriptionForFailed>{failed_links}</descriptionForFailed>
> + <setForMatrix>false</setForMatrix>
> </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
> </publishers>
> <buildWrappers/>
> </project>
> """.format(board=board, reboot=str(reboot), rebuild=str(rebuild),
> precleanup=str(precleanup), postcleanup=str(postcleanup),
> testdir=str(test), testname=str(test).split('.')[1],
> - testplan=testplan, timeout=timeout, flot_link=flot_link,
> html_links=html_links))
> + testplan=testplan, timeout=timeout, flot_link=flot_link,
> html_links=html_links, failed_links=failed_links))
> fd.close()
>
> print("Creating job " + test)
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs
2017-03-30 1:04 ` [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs Daniel Sangorrin
@ 2017-03-30 23:06 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 23:06 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
OK.
Acked-by: Tim Bird
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, March 29, 2017 6:05 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc | 36 +++++++++++++++++++++---------------
> 1 file changed, 21 insertions(+), 15 deletions(-)
>
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index 82a8c18..372c32a 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -85,7 +85,7 @@ TARGET_ENV_VAR="FTC_TARGET"
> # format for command_help mapping with: key=name, value=(summary,
> long description)
> command_help = {
> "add-jobs":("Adds jobs to Jenkins.",
> - """Usage: ftc add-jobs -b <target> [-p <testplan> | -t <testcase>] [-d
> <foo.dist>]
> + """Usage: ftc add-jobs -b <target> [-p <testplan> | -t <testcase> -s
> <testspec>]
> Example: sudo -u jenkins ftc add-jobs -b docker -p testplan_docker
> Example: sudo -u jenkins ftc add-jobs -b docker -t Benchmark.Dhrystone
> Use list-plans to see the available test plans.
> @@ -93,13 +93,13 @@ command_help = {
> This interface may change in the future."""),
>
> "rm-jobs":("Removes jobs from Jenkins.",
> - """Usage: ftc rm-jobs <target>.<testplan>.<testcase>
> + """Usage: ftc rm-jobs <target>.<testspec>.<testtype>.<testcase>
> Use list-jobs to see the existing jobs. A wildcard can be used to
> specify which jobs to remove (just make sure you have 4 words):
> Example: sudo -u jenkins ftc rm-jobs docker.testplan_docker.*.*
> Example: sudo -u jenkins ftc rm-jobs docker.*.F*.*stress
> - Multiple combinations of the <target>.<testplan>.<testcase> pattern
> - can be passed as well.
> + Multiple combinations of the <target>.<testspec>.<testtype>.<testcase>
> + pattern can be passed as well.
>
> If no option is provided all existing jobs will be removed."""),
>
> @@ -741,7 +741,7 @@ def get_includes(include_filename, conf):
> inc_vars = parse_shell_file(inc_path, conf)
> return inc_vars
>
> -def create_job(board, testplan, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks):
> +def create_job(board, testspec, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks):
> # flot only necessary for Benchmarks
> if str(test).split('.')[0] == 'Benchmark':
> flot_link = '<flotile.FlotPublisher plugin="flot@1.0-SNAPSHOT"/>'
> @@ -781,7 +781,7 @@ export Target_PreCleanup={precleanup}
> export Target_PostCleanup={postcleanup}
> export TESTDIR={testdir}
> export TESTNAME={testname}
> -export TESTPLAN="{testplan}"
> +export TESTSPEC="{testspec}"
> timeout --signal=9 {timeout} /bin/bash
> $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
> </command>
> </hudson.tasks.Shell>
> @@ -800,13 +800,13 @@ timeout --signal=9 {timeout} /bin/bash
> $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{
> </project>
> """.format(board=board, reboot=str(reboot), rebuild=str(rebuild),
> precleanup=str(precleanup), postcleanup=str(postcleanup),
> testdir=str(test), testname=str(test).split('.')[1],
> - testplan=testplan, timeout=timeout, flot_link=flot_link,
> html_links=html_links, failed_links=failed_links))
> + testspec=testspec, timeout=timeout, flot_link=flot_link,
> html_links=html_links, failed_links=failed_links))
> fd.close()
>
> print("Creating job " + test)
> try:
> subprocess.call('java -jar /var/cache/jenkins/war/WEB-INF/jenkins-
> cli.jar -s http://localhost:8080/fuego create-job ' +
> - board + '.' + testplan + '.' + str(test) + ' < ' + tmp, shell=True)
> + board + '.' + testspec + '.' + str(test) + ' < ' + tmp, shell=True)
> subprocess.call('rm -f ' + tmp, shell=True)
> except Exception as e:
> print("Job already exists")
> @@ -862,6 +862,7 @@ def create_batch_job(board, testplan, alltests):
> def parse_testplan(testplan):
> abspath = '/fuego-core/engine/overlays/testplans/' + testplan + '.json'
> alltests = []
> + alltestspecs = []
> alltimeouts = []
> allextralinks = []
> allreboots = []
> @@ -873,6 +874,7 @@ def parse_testplan(testplan):
> plan = json.load(f)
> for test in plan['tests']:
> alltests.append(test['testName'])
> + alltestspecs.append(test['spec'])
> if 'timeout' in test:
> alltimeouts.append(test['timeout'])
> else:
> @@ -897,7 +899,7 @@ def parse_testplan(testplan):
> allextralinks.append(test['extralinks'])
> else:
> allextralinks.append('None')
> - return alltests, alltimeouts, allreboots, allrebuilds, allprecleanups,
> allpostcleanups, allextralinks
> + return alltests, alltestspecs, alltimeouts, allreboots, allrebuilds,
> allprecleanups, allpostcleanups, allextralinks
>
> def do_add_jobs(conf, options):
> if '-b' in options:
> @@ -925,8 +927,12 @@ def do_add_jobs(conf, options):
> raise Exception('Test %s not found.' % test)
> options.remove('-t')
> options.remove(test)
> - # TODO: use spec instead of testplan_default
> - testplan = 'testplan_default'
> + if '-s' in options:
> + spec = options[options.index('-s') + 1]
> + options.remove('-s')
> + options.remove(spec)
> + else:
> + spec = 'default'
> else:
> raise Exception('No testplan or testcase supplied.')
>
> @@ -937,11 +943,11 @@ def do_add_jobs(conf, options):
> precleanup = 'true'
> postcleanup = 'true'
> jobextralinks = 'None'
> - create_job(board, testplan, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks)
> + create_job(board, spec, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks)
> else:
> - alltests, alltimeouts, allreboots, allrebuilds, allprecleanups,
> allpostcleanups, allextralinks = parse_testplan(testplan)
> - for test, timeout, reboot, rebuild, precleanup, postcleanup,
> jobextralinks in zip(alltests, alltimeouts, allreboots, allrebuilds,
> allprecleanups, allpostcleanups, allextralinks):
> - create_job(board, testplan, test, timeout, reboot, rebuild,
> precleanup, postcleanup, jobextralinks)
> + alltests, alltestspecs, alltimeouts, allreboots, allrebuilds, allprecleanups,
> allpostcleanups, allextralinks = parse_testplan(testplan)
> + for test, spec, timeout, reboot, rebuild, precleanup, postcleanup,
> jobextralinks in zip(alltests, alltestspecs, alltimeouts, allreboots, allrebuilds,
> allprecleanups, allpostcleanups, allextralinks):
> + create_job(board, spec, test, timeout, reboot, rebuild, precleanup,
> postcleanup, jobextralinks)
> create_batch_job(board, testplan, alltests)
> sys.exit(0)
>
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths
2017-03-30 1:04 ` [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths Daniel Sangorrin
@ 2017-03-30 23:13 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 23:13 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
Thanks. I applied the patch.
The whole approach of ftc-unit-test.sh needs to be revisited.
Jenkins nodes are not just entries in Jenkins' config.xml, and since
the board directory is read-only, a different approach will be needed
to create the ftc-test board.
Do you think I should just put it there, and leave it there, for testing
purposes? It won't show up as a board unless a Fuego user does an
add-node with it, and then it will be available to do things like:
add-node, rm-node, get, set and other tests with.
Let me know what you think.
For now,
Acked-by: Tim Bird
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, March 29, 2017 6:05 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths
>
> Note: this is untested and probably won't work but
> it's closer to working :)
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/ftc-unit-test.sh | 21 ++-------------------
> 1 file changed, 2 insertions(+), 19 deletions(-)
>
> diff --git a/engine/scripts/ftc-unit-test.sh b/engine/scripts/ftc-unit-test.sh
> index b7d910a..911888e 100755
> --- a/engine/scripts/ftc-unit-test.sh
> +++ b/engine/scripts/ftc-unit-test.sh
> @@ -2,7 +2,7 @@
> #
> # this script performs a unit test of ftc
> #
> -TEST_BOARD_FILE=/userdata/conf/boards/ftc-test.board
> +TEST_BOARD_FILE=/fuego-ro/boards/ftc-test.board
> PATCH_FILE=add-ftc-test-node.patch
>
> #set -x
> @@ -100,26 +100,9 @@ cat <<'EOFP' > $PATCH_FILE
> + <mode>NORMAL</mode>
> + <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
> + <launcher class="hudson.slaves.CommandLauncher">
> -+ <agentCommand>java -jar
> /home/jenkins/slave.jar</agentCommand>
> ++ <agentCommand>java -jar /fuego-
> core/engine/slave.jar</agentCommand>
> + </launcher>
> + <label></label>
> -+ <nodeProperties>
> -+ <hudson.slaves.EnvironmentVariablesNodeProperty>
> -+ <envVars serialization="custom">
> -+ <unserializable-parents/>
> -+ <tree-map>
> -+ <default>
> -+ <comparator class="hudson.util.CaseInsensitiveComparator"/>
> -+ </default>
> -+ <int>2</int>
> -+ <string>BOARD_OVERLAY</string>
> -+ <string>boards/ftc-test.board</string>
> -+ <string>DISTRIB</string>
> -+ <string>distribs/nologger.dist</string>
> -+ </tree-map>
> -+ </envVars>
> -+ </hudson.slaves.EnvironmentVariablesNodeProperty>
> -+ </nodeProperties>
> + <userId>anonymous</userId>
> + </slave>
> </slaves>
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 15/16] fail_check_cases should not abort the job
2017-03-30 1:04 ` [Fuego] [PATCH 15/16] fail_check_cases should not abort the job Daniel Sangorrin
@ 2017-03-30 23:16 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 23:16 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
OK. Thanks.
I agree that the error should be reflected into FUEGO_RESULT somehow.
This is a bit of a weird test, looking in the syslog for oopses and such. This is
probably a nice safety mechanism for kernel and driver testing, to detect
when some operation caused an oops, so it's worth keeping them around.
Acked-by: Tim Bird
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, March 29, 2017 6:05 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 15/16] fail_check_cases should not abort the job
>
> failure is something normal and should be kept in a variable
> to return it at the end to jenkins. If we just abort then
> post_test will not have a chance to return $FUEGO_RESULT
>
> TODO: probably we should change $FUEGO_RESULT though
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 9 ++++-----
> 1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index 2483f61..4379271 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -331,7 +331,6 @@ function bench_processing {
> }
>
> # search in test log for {!JOB_NAME}_FAIL_PATTERN_n fail cases and abort
> with message {!JOB_NAME}_FAIL_MESSAGE_n if found
> -# args: $1 - path to test log
> function fail_check_cases () {
> testlog="${LOGDIR}/testlog.txt"
> slog_prefix="${LOGDIR}/syslog"
> @@ -364,8 +363,7 @@ function fail_check_cases () {
>
> if diff -ua ${slog_prefix}.before ${slog_prefix}.after | grep -vEf
> "$FUEGO_CORE/engine/scripts/syslog.ignore" | grep -E -e $fptemplate;
> then
> - echo "Located failing message in syslog diff"
> - abort_job "Detected fail message in syslog diff: $fpmessage"
> + echo "Detected fail message in syslog diff: $fpmessage"
> else
> continue
> fi
> @@ -373,8 +371,7 @@ function fail_check_cases () {
>
> if grep -e "$fptemplate" $testlog ;
> then
> - echo "Located failing message in $1"
> - abort_job "Detected fail message: $fpmessage"
> + echo "Detected fail message in $testlog: $fpmessage"
> fi
> done
> }
> @@ -444,8 +441,10 @@ function post_test {
> ov_logger "Test $1 is finished"
>
> # Syslog comparison
> + # TODO: should affect FUEGO_RESULT
> syslog_cmp
>
> + # TODO: should affect FUEGO_RESULT
> fail_check_cases || true
>
> # create functional result file
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [Fuego] [PATCH 16/16] a bit of cleaning and style fixes
2017-03-30 1:04 ` [Fuego] [PATCH 16/16] a bit of cleaning and style fixes Daniel Sangorrin
@ 2017-03-30 23:19 ` Bird, Timothy
0 siblings, 0 replies; 33+ messages in thread
From: Bird, Timothy @ 2017-03-30 23:19 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
OK.
Acked-by: Tim Bird
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, March 29, 2017 6:05 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 16/16] a bit of cleaning and style fixes
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> engine/scripts/functions.sh | 2 +-
> engine/scripts/ovgen.py | 17 ++---------------
> 2 files changed, 3 insertions(+), 16 deletions(-)
>
> diff --git a/engine/scripts/functions.sh b/engine/scripts/functions.sh
> index 4379271..e499ab4 100755
> --- a/engine/scripts/functions.sh
> +++ b/engine/scripts/functions.sh
> @@ -500,7 +500,7 @@ function log_compare {
> return $FUEGO_RESULT
> fi
>
> - cd ${LOGDIR}
> + cd ${LOGDIR}
> LOGFILE="testlog.txt"
> PARSED_LOGFILE="testlog.${4}.txt"
>
> diff --git a/engine/scripts/ovgen.py b/engine/scripts/ovgen.py
> index 10d1cf4..4b25d86 100755
> --- a/engine/scripts/ovgen.py
> +++ b/engine/scripts/ovgen.py
> @@ -37,7 +37,6 @@ OFVAR_DESCRIPTION="DESCRIPTION"
> LVAR_NAME="NAME"
> LVAR_DESCRIPTION="DESCRIPTION"
>
> -
> class OFClass:
> def __init__(self):
> self.name=""
> @@ -65,7 +64,6 @@ class OFLayer:
> def __str__(self):
> return "<OFLayer name:\"%s\" descr:\"%s\" vars:%s>\n" % (self.name,
> self.description, self.vars)
>
> -
> class TestSpecs:
> def __init__(self):
> self.name=""
> @@ -108,7 +106,6 @@ class SpecException(Exception):
> def __str__(self):
> return repr(self.value)
>
> -
> def debug_print(string, lev=1):
> if lev <= log_lvl:
> print "log: " + string
> @@ -130,7 +127,6 @@ def parseOFVars(line, ofc):
>
> return True
>
> -
> # parse variables definitions
> def parseVars(line, ofc):
> m = re.search("(\w+)=\"(.*)\"", line)
> @@ -164,7 +160,6 @@ def parseFunctionBody(name, f):
>
> return funcbody
>
> -
> def parseFunction(line, f):
> m = re.search("\s?function (\w+).*{", line)
> if m==None:
> @@ -209,7 +204,6 @@ def parseBaseFile(baseFilePath, ofc):
> elif baseParseFunction(line, f, ofc):
> debug_print("function found", 3)
>
> -
> # parse all base files in dir
> def parseBaseDir(baseDirPath, ofcls):
> debug_print ("\n------------\nparsing " + baseDirPath + " fuegoclass dir
> ...\n")
> @@ -221,13 +215,12 @@ def parseBaseDir(baseDirPath, ofcls):
> debug_print ("parsed %s class\n------------\n" % (ofc.name))
> ofcls[ofc.name]=(ofc)
>
> -
> def parseInherit(line, ofcls):
> m = re.search("inherit \"(.+)\"", line)
> - if m==None: return None
> + if m == None:
> + return None
> clname = m.group(1)
>
> -
> if clname not in ofcls:
> raise OFClassNotFoundException("No such class: %s" % (clname))
>
> @@ -259,7 +252,6 @@ def parseLayerVarOverride(line, layer, inhclass):
>
> return True
>
> -
> def parseLayerFuncOverride(line, layer, inhclass, f):
> m = re.search("override-func (\w+)(.*)", line)
> if m==None: return False
> @@ -322,7 +314,6 @@ def parseLayerCapList(line, layer, inhclass):
>
> return True
>
> -
> def parseOverrideFile(overrideFile, layer, ofcls):
> debug_print ("\n-----------\nparsing %s override ...\n" % (overrideFile))
> f = open(overrideFile)
> @@ -472,7 +463,6 @@ def parseSpec(testdir, testspec):
>
> def run(test_args=None):
> parser = argparse.ArgumentParser(description='Read OF class files,
> override files and generate prolog defining all variables and functions')
> -
> parser.add_argument('--classdir', help='OF base class directory',
> required=True)
> parser.add_argument('--ovfiles', nargs='+', metavar='OVFILE', help='list of
> directories containing .override files', required=True)
> parser.add_argument('--testdir', help='e.g.: Benchmark.Dhrystone',
> required=True)
> @@ -483,11 +473,9 @@ def run(test_args=None):
> args = parser.parse_args(args=test_args)
>
> if args.debug:
> -
> if args.debug < 1 or args.debug > 3:
> print "Error: wrong debug lvl: %s" % (args.debug)
> sys.exit
> -
> global log_lvl
> log_lvl = args.debug
>
> @@ -497,7 +485,6 @@ def run(test_args=None):
> parseBaseDir(args.classdir, ofcls)
>
> layers = {}
> -
> classes = []
> for ovf in args.ovfiles:
> classes = classes + parseOverrideFile(ovf, layers, ofcls)
> --
> 2.7.4
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2017-03-30 23:19 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-30 1:04 [Fuego] fuego-core patches (migration to testspecs and more) Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 01/16] indentation: fix indentation for dhrystone parser Daniel Sangorrin
2017-03-30 1:04 ` [Fuego] [PATCH 02/16] abort: fix the abort function Daniel Sangorrin
2017-03-30 20:26 ` Bird, Timothy
2017-03-30 20:29 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 03/16] shell e flag: remove any e flag from fuego Daniel Sangorrin
2017-03-30 20:40 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 04/16] parser: remove error message when no matches Daniel Sangorrin
2017-03-30 20:51 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 05/16] benchmark: use the same pattern as in functional Daniel Sangorrin
2017-03-30 20:54 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 06/16] board file: remove conf from the path to the board file Daniel Sangorrin
2017-03-30 21:24 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 07/16] batch: remove batch_testplan reference Daniel Sangorrin
2017-03-30 21:32 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 08/16] testplans: use testspecs instead of testplans internally Daniel Sangorrin
2017-03-30 21:43 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 09/16] distrib: DISTRIB must be defined in the board file Daniel Sangorrin
2017-03-30 22:46 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 10/16] user_checks: add to commands so they execute under jenkins Daniel Sangorrin
2017-03-30 22:49 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 11/16] ftc:rm-jobs: remove limitation of 4 words Daniel Sangorrin
2017-03-30 22:50 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 12/16] description setter: put log if the test fails Daniel Sangorrin
2017-03-30 23:04 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 13/16] ftc: remove testplans and use testspecs Daniel Sangorrin
2017-03-30 23:06 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 14/16] ftc test: remove distrib and update paths Daniel Sangorrin
2017-03-30 23:13 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 15/16] fail_check_cases should not abort the job Daniel Sangorrin
2017-03-30 23:16 ` Bird, Timothy
2017-03-30 1:04 ` [Fuego] [PATCH 16/16] a bit of cleaning and style fixes Daniel Sangorrin
2017-03-30 23:19 ` Bird, Timothy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.