[Fuego] [PATCH] ptest: add a wrapper for the ptest-runner

* [Fuego] [PATCH] ptest: add a wrapper for the ptest-runner
@ 2019-03-13  9:19 Daniel Sangorrin
  2019-03-14  1:27 ` Tim.Bird
  0 siblings, 1 reply; 2+ messages in thread
From: Daniel Sangorrin @ 2019-03-13  9:19 UTC (permalink / raw)
  To: fuego

This test assumes that your board has a yocto/oe filesystem
and ptests are installed:
https://wiki.yoctoproject.org/wiki/Ptest#ptest-runner_Usage
DISTRO_FEATURES_append = " ptest"
EXTRA_IMAGE_FEATURES += "ptest-pkgs"

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 tests/Functional.ptest/chart_config.json |  3 +++
 tests/Functional.ptest/fuego_test.sh     | 13 +++++++++++
 tests/Functional.ptest/parser.py         | 38 ++++++++++++++++++++++++++++++++
 tests/Functional.ptest/spec.json         | 10 +++++++++
 tests/Functional.ptest/test.yaml         | 24 ++++++++++++++++++++
 5 files changed, 88 insertions(+)
 create mode 100644 tests/Functional.ptest/chart_config.json
 create mode 100755 tests/Functional.ptest/fuego_test.sh
 create mode 100755 tests/Functional.ptest/parser.py
 create mode 100644 tests/Functional.ptest/spec.json
 create mode 100644 tests/Functional.ptest/test.yaml

diff --git a/tests/Functional.ptest/chart_config.json b/tests/Functional.ptest/chart_config.json
new file mode 100644
index 0000000..1849992
--- /dev/null
+++ b/tests/Functional.ptest/chart_config.json
@@ -0,0 +1,3 @@
+{
+    "chart_type": "testset_summary_table"
+}
diff --git a/tests/Functional.ptest/fuego_test.sh b/tests/Functional.ptest/fuego_test.sh
new file mode 100755
index 0000000..13cda0d
--- /dev/null
+++ b/tests/Functional.ptest/fuego_test.sh
@@ -0,0 +1,13 @@
+# Usage: ptest-runner [-d directory] [-l list] [-t timeout] [-x xml-filename] [-h] [ptest1 ptest2 ...]
+
+function test_pre_check {
+    assert_has_program ptest-runner
+    AVAILABLE_TESTS=$(cmd "ptest-runner -l | tail -n +2 | awk '{print \$1;}'")
+    TESTS=${FUNCTIONAL_PTEST_TESTS:-$AVAILABLE_TESTS}
+    echo "The following tests will be run: $TESTS"
+}
+
+function test_run {
+    report "ptest-runner $TESTS"
+}
+
diff --git a/tests/Functional.ptest/parser.py b/tests/Functional.ptest/parser.py
new file mode 100755
index 0000000..c5690c4
--- /dev/null
+++ b/tests/Functional.ptest/parser.py
@@ -0,0 +1,38 @@
+#!/usr/bin/python
+# See common.py for description of command-line arguments
+
+import os, sys, collections
+import common as plib
+import re
+
+measurements = {}
+measurements = collections.OrderedDict()
+
+with open(plib.TEST_LOG) as f:
+    print "Reading current values from " + plib.TEST_LOG +"\n"
+    lines = f.readlines()
+
+for line in lines:
+    line = line.strip()
+    if line.startswith('BEGIN:'):
+        # BEGIN: /usr/lib/gzip/ptest <-- we want gzip
+        test_set = line.split(':')[1].strip().split('/')[-2]
+        continue
+    elif line.startswith('PASS:') or line.startswith('XPASS:'):
+        # PASS: zlib shared <-- we want zlib_shared
+        test_case = line.split(':')[1].strip().replace(' ', '_')
+        result = 'PASS'
+    elif line.startswith('SKIP:'):
+        test_case = line.split(':')[1].strip().replace(' ', '_')
+        result = 'SKIP'
+    elif line.startswith('FAIL:') or line.startswith('XFAIL:') or line.startswith('ERROR:'):
+        test_case = line.split(':')[1].strip().replace(' ', '_')
+        result = 'FAIL'
+    elif line.startswith('END:'):
+        continue
+    else:
+        # TODO: split output per test here
+        continue
+    measurements[test_set + '.' + test_case] = result
+
+sys.exit(plib.process(measurements))
diff --git a/tests/Functional.ptest/spec.json b/tests/Functional.ptest/spec.json
new file mode 100644
index 0000000..290adc4
--- /dev/null
+++ b/tests/Functional.ptest/spec.json
@@ -0,0 +1,10 @@
+{
+    "testName": "Functional.ptest",
+    "specs": {
+        "default": {
+        },
+        "smoke": {
+            "TESTS": "zlib gzip attr"
+        }
+    }
+}
diff --git a/tests/Functional.ptest/test.yaml b/tests/Functional.ptest/test.yaml
new file mode 100644
index 0000000..9d13ff6
--- /dev/null
+++ b/tests/Functional.ptest/test.yaml
@@ -0,0 +1,24 @@
+fuego_package_version: 1
+name: Functional.ptest
+description: |
+    Ptest (package test) is the test framework adopted by the Yocto project
+license: GPL-2.0
+author: Aníbal Limón
+maintainer: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
+version: undefined
+fuego_release: 1
+type: Functional
+tags: ['ptest', 'yocto', 'poky', 'package']
+git_src: https://git.yoctoproject.org/git/ptest-runner2
+params:
+    - TESTS:
+        description: Tests to run
+        example: "zlib gzip attr"
+        default: all available tests (ptest-runner -l)
+        optional: yes
+data_files:
+    - chart_config.json
+    - fuego_test.sh
+    - parser.py
+    - spec.json
+    - test.yaml
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 2+ messages in thread