All of lore.kernel.org
 help / color / mirror / Atom feed
* [Fuego] LTP: changes to the auto skip feature
@ 2018-05-18  2:48 Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 01/11] LTP: skip sbrk03 and others depending on architecture Daniel Sangorrin
                   ` (11 more replies)
  0 siblings, 12 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Hi Tim,

I made a few changes to the auto skip feature that I added to LTP
a while ago.

Patches: 
[PATCH 01/11] LTP: skip sbrk03 and others depending on architecture
[PATCH 02/11] LTP: do not skip openat02
[PATCH 03/11] LTP: fix small errata
[PATCH 04/11] LTP: do not skip addkey02 and add a doc file
[PATCH 05/11] LTP: do not skip adjtimex02
[PATCH 06/11] LTP: add a kernel version check using version_lt
[PATCH 07/11] LTP: skip fanotify07 depending on the kernel version
[PATCH 08/11] LTP: add noautoskip option for power users
[PATCH 09/11] LTP: add noautoskip to the quickhit spec
[PATCH 10/11] LTP: add test.yaml describing all variables
[PATCH 11/11] LTP: add doc about fcntl35

Stats:
 engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.add_key02.ftmp | 62 ++++++++++++++++++++++++++++
 engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.fcntl35.ftmp   | 53 ++++++++++++++++++++++++
 engine/tests/Functional.LTP/fuego_test.sh                               | 89 +++++++++++++++++++++++-----------------
 engine/tests/Functional.LTP/spec.json                                   |  1 +
 engine/tests/Functional.LTP/test.yaml                                   | 85 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 253 insertions(+), 37 deletions(-)

Thanks,
Daniel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 01/11] LTP: skip sbrk03 and others depending on architecture
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 02/11] LTP: do not skip openat02 Daniel Sangorrin
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

The test cases sbrk03, switch01, ptrace04, cacheflush01 depend
on the target's architecture so only skip them if the architecture
is different.
Note: not sure the names for this architectures is standard.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index eef3c0a..748ad68 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -181,10 +181,6 @@ function test_pre_check {
     echo "Tests skipped by default in Fuego for now"
     echo "# skip these test cases" > ${LOGDIR}/skiplist.txt
     skip_tests "su01" # too complicated to setup
-    skip_tests "sbrk03" # Only works in 32bit on s390 series system
-    skip_tests "switch01" # Requires a 64-bit processor that supports little-endian mode,such as POWER6.
-    skip_tests "ptrace04" # Requires blackfin processor
-    skip_tests "cacheflush01" # Available only on MIPS-based systems
     skip_tests "bdflush01" # bdflush was deprecated by pdflush and pdflush removed from the kernel (https://lwn.net/Articles/508212/)
     skip_tests "fcntl06" # Linux kernel doesn't implement R_GETLK/R_SETLK
     skip_tests "fcntl06_64" # Linux kernel doesn't implement R_GETLK/R_SETLK
@@ -244,6 +240,22 @@ function test_pre_check {
     skip_if_kconfig_differs "CONFIG_SGETMASK_SYSCALL=y" "ssetmask01"
 
     echo "Tests skipped depending on the architecture"
+    if [ "$ARCHITECTURE" != "s390" ]; then
+        skip_tests "sbrk03" # Only works in 32bit on s390 series system
+    fi
+
+    if [ "$ARCHITECTURE" != "powerpc" ]; then
+        skip_tests "switch01" # Requires a 64-bit processor that supports little-endian mode,such as POWER6.
+    fi
+
+    if [ "$ARCHITECTURE" != "blackfin" ]; then
+        skip_tests "ptrace04" # Requires blackfin processor
+    fi
+
+    if [ "$ARCHITECTURE" != "mips" ]; then
+        skip_tests "cacheflush01" # Available only on MIPS-based systems
+    fi
+
     if [ "$ARCHITECTURE" = "x86_64" ]; then
         skip_tests "chown01_16 chown02_16 chown03_16 chown04_16 chown05_16"
         skip_tests "fchown01_16 fchown02_16 fchown03_16 fchown04_16 fchown05_16"
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 02/11] LTP: do not skip openat02
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 01/11] LTP: skip sbrk03 and others depending on architecture Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 03/11] LTP: fix small errata Daniel Sangorrin
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

openat02 already detects the target filesystem flags
configuration and results in a TCONF with an easy to
understand message, so do not skip it automatically.
In fact there are several sub-tests inside openat02
that are also worth it.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 748ad68..d326345 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -185,7 +185,6 @@ function test_pre_check {
     skip_tests "fcntl06" # Linux kernel doesn't implement R_GETLK/R_SETLK
     skip_tests "fcntl06_64" # Linux kernel doesn't implement R_GETLK/R_SETLK
     skip_tests "munlockall02" # munlockall can be called form user space
-    skip_tests "openat02" # Requires a filesystem with noatime and relatime
     skip_tests "utimensat01" # kernel patch f2b20f6ee842313a changed return value from -EACCESS to EPERM when a file is immutable and update time is NULL or UTIME_NOW
 
     echo "Tests skipped if the target /tmp folder is mounted on tmpfs"
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 03/11] LTP: fix small errata
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 01/11] LTP: skip sbrk03 and others depending on architecture Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 02/11] LTP: do not skip openat02 Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 04/11] LTP: do not skip addkey02 and add a doc file Daniel Sangorrin
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index d326345..222aa67 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -184,7 +184,7 @@ function test_pre_check {
     skip_tests "bdflush01" # bdflush was deprecated by pdflush and pdflush removed from the kernel (https://lwn.net/Articles/508212/)
     skip_tests "fcntl06" # Linux kernel doesn't implement R_GETLK/R_SETLK
     skip_tests "fcntl06_64" # Linux kernel doesn't implement R_GETLK/R_SETLK
-    skip_tests "munlockall02" # munlockall can be called form user space
+    skip_tests "munlockall02" # munlockall can be called from user space
     skip_tests "utimensat01" # kernel patch f2b20f6ee842313a changed return value from -EACCESS to EPERM when a file is immutable and update time is NULL or UTIME_NOW
 
     echo "Tests skipped if the target /tmp folder is mounted on tmpfs"
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 04/11] LTP: do not skip addkey02 and add a doc file
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (2 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 03/11] LTP: fix small errata Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 05/11] LTP: do not skip adjtimex02 Daniel Sangorrin
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 .../docs/Functional.LTP.syscalls.add_key02.ftmp    | 62 ++++++++++++++++++++++
 engine/tests/Functional.LTP/fuego_test.sh          |  1 -
 2 files changed, 62 insertions(+), 1 deletion(-)
 create mode 100644 engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.add_key02.ftmp

diff --git a/engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.add_key02.ftmp b/engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.add_key02.ftmp
new file mode 100644
index 0000000..20cd42f
--- /dev/null
+++ b/engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.add_key02.ftmp
@@ -0,0 +1,62 @@
+===========
+Description
+===========
+
+Obtained from addkey02.c DESCRIPTION:
+	Test that the add_key() syscall correctly handles a NULL payload with nonzero
+	length.  Specifically, it should fail with EFAULT rather than oopsing the
+	kernel with a NULL pointer dereference or failing with EINVAL, as it did
+	before (depending on the key type).  This is a regression test for commit
+	5649645d725c ("KEYS: fix dereferencing NULL payload with nonzero length").
+	
+	Note that none of the key types that exhibited the NULL pointer dereference
+	are guaranteed to be built into the kernel, so we just test as many as we
+	can, in the hope of catching one.  We also test with the "user" key type for
+	good measure, although it was one of the types that failed with EINVAL rather
+	than dereferencing NULL.
+	
+	This has been assigned CVE-2017-15274.
+
+Other notes:
+	Commit 5649645d725c appears to have been included since the kernel 4.12.
+
+====
+Tags
+====
+
+* kernel, syscall, addkey
+
+=========
+Resources
+=========
+
+* https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5649645d725c73df4302428ee4e02c869248b4c5
+
+=======
+Results
+=======
+
+Running on a PC (64 bits) using Debian Jessie (kernel 3.16):
+  add_key02.c:81: CONF: kernel doesn't support key type 'asymmetric'
+  add_key02.c:81: CONF: kernel doesn't support key type 'cifs.idmap'
+  add_key02.c:81: CONF: kernel doesn't support key type 'cifs.spnego'
+  add_key02.c:81: CONF: kernel doesn't support key type 'pkcs7_test'
+  add_key02.c:81: CONF: kernel doesn't support key type 'rxrpc'
+  add_key02.c:81: CONF: kernel doesn't support key type 'rxrpc_s'
+  add_key02.c:96: FAIL: unexpected error with key type 'user': EINVAL
+  add_key02.c:96: FAIL: unexpected error with key type 'logon': EINVAL
+
+The kernel should have returned an EFAULT error, not EINVAL:
+  Ref: https://github.com/linux-test-project/ltp/issues/182
+
+.. fuego_result_list::
+
+======
+Status
+======
+
+.. fuego_status::
+
+=====
+Notes
+=====
diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 222aa67..1baffc8 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -205,7 +205,6 @@ function test_pre_check {
     echo "kernel version: $kver"
     skip_tests "adjtimex02" # needs a kernel that doesn't normalize buf.offset if it is outside the acceptable range
     skip_tests "fallocate04" # needs Linux 3.15 or newer
-    skip_tests "add_key02" # needs a kernel with the 5649645d725c commit
     skip_tests "inotify06" # will loop/crash kernels that dont have commit 8f2f3eb59dff (<4.2)
     skip_tests "ptrace03" # Only works for <2.6.26, above that the kernel allows to trace init
 
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 05/11] LTP: do not skip adjtimex02
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (3 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 04/11] LTP: do not skip addkey02 and add a doc file Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 06/11] LTP: add a kernel version check using version_lt Daniel Sangorrin
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

This test may only be interesting for some kernels but
that is not reason to automatically skip it.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 1baffc8..aa166da 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -203,7 +203,6 @@ function test_pre_check {
     # FIXTHIS: compare with the kernel version before adding tests to the skiplist
     kver=$(cmd "uname -r")
     echo "kernel version: $kver"
-    skip_tests "adjtimex02" # needs a kernel that doesn't normalize buf.offset if it is outside the acceptable range
     skip_tests "fallocate04" # needs Linux 3.15 or newer
     skip_tests "inotify06" # will loop/crash kernels that dont have commit 8f2f3eb59dff (<4.2)
     skip_tests "ptrace03" # Only works for <2.6.26, above that the kernel allows to trace init
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 06/11] LTP: add a kernel version check using version_lt
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (4 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 05/11] LTP: do not skip adjtimex02 Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 07/11] LTP: skip fanotify07 depending on the kernel version Daniel Sangorrin
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 35 +++++++++++++++++++++++++------
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index aa166da..82fafad 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -110,6 +110,31 @@ function skip_if_command_unavailable {
     fi
 }
 
+# $1: minimum kernel version
+# $2: string with LTP test names separated by spaces
+# FIXTHIS: do not skip tests by kernel version if the kernel is an LTS kernel
+function skip_if_kver_lt {
+    local MIN_KVER="$1"
+    local TESTS="$2"
+    local KVER=$(cmd "uname -r")
+
+    if version_lt "${KVER}" "${MIN_KVER}"; then
+        skip_tests "${TESTS}"
+    fi
+}
+
+# $1: maximum kernel version
+# $2: string with LTP test names separated by spaces
+function skip_if_kver_gt {
+    local MAX_KVER="$1"
+    local TESTS="$2"
+    local KVER=$(cmd "uname -r")
+
+    if version_lt "${MAX_KVER}" "${KVER}"; then
+        skip_tests "${TESTS}"
+    fi
+}
+
 # $1: a HAVE_xxx option to grep from ltp's include/config.h (after ./configure)
 # $2: string with LTP test names separated by spaces
 # $3: optional message
@@ -200,12 +225,10 @@ function test_pre_check {
     rm -f $target_mounts
 
     echo "Tests skipped by Linux kernel version"
-    # FIXTHIS: compare with the kernel version before adding tests to the skiplist
-    kver=$(cmd "uname -r")
-    echo "kernel version: $kver"
-    skip_tests "fallocate04" # needs Linux 3.15 or newer
-    skip_tests "inotify06" # will loop/crash kernels that dont have commit 8f2f3eb59dff (<4.2)
-    skip_tests "ptrace03" # Only works for <2.6.26, above that the kernel allows to trace init
+    # FIXTHIS: grep tst_kvercmp on LTP to check for other tests that require specific versions
+    skip_if_kver_gt "2.6.26" "ptrace03" # Only works for <2.6.26, above that the kernel allows to trace init
+    skip_if_kver_lt "3.15" "fallocate04" # The test needs Linux 3.15 or newer
+    skip_if_kver_lt "4.2" "inotify06" # will loop/crash kernels that dont have commit 8f2f3eb59dff (<4.2)
 
     echo "Tests skipped depending on the availability of a command on the target"
     # FIXTHIS: only check the necessary ones
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 07/11] LTP: skip fanotify07 depending on the kernel version
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (5 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 06/11] LTP: add a kernel version check using version_lt Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 08/11] LTP: add noautoskip option for power users Daniel Sangorrin
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 82fafad..185b4cc 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -229,6 +229,7 @@ function test_pre_check {
     skip_if_kver_gt "2.6.26" "ptrace03" # Only works for <2.6.26, above that the kernel allows to trace init
     skip_if_kver_lt "3.15" "fallocate04" # The test needs Linux 3.15 or newer
     skip_if_kver_lt "4.2" "inotify06" # will loop/crash kernels that dont have commit 8f2f3eb59dff (<4.2)
+    skip_if_kver_lt "4.14" "fanotify07" # see Functional.LTP/docs/Functional.LTP.syscalls.fanotify07.ftmp
 
     echo "Tests skipped depending on the availability of a command on the target"
     # FIXTHIS: only check the necessary ones
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 08/11] LTP: add noautoskip option for power users
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (6 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 07/11] LTP: skip fanotify07 depending on the kernel version Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 09/11] LTP: add noautoskip to the quickhit spec Daniel Sangorrin
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 185b4cc..44296c5 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -84,6 +84,11 @@ func/sched_jitter
 # $1: string with tests to skip separated by spaces
 # $2: absolute path to the skipfile (default: ${LOGDIR}/skiplist.txt)
 function skip_tests {
+    if [ -n "${FUNCTIONAL_LTP_NOAUTOSKIP}" ]; then
+        echo "Not skipping tests: ${TESTS[@]}."
+        return 0
+    fi
+
     # split $1 on whitespace, without file globbing
     set -f
     local TESTS=($1)
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 09/11] LTP: add noautoskip to the quickhit spec
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (7 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 08/11] LTP: add noautoskip option for power users Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 10/11] LTP: add test.yaml describing all variables Daniel Sangorrin
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

This is mainly for visibility.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/spec.json | 1 +
 1 file changed, 1 insertion(+)

diff --git a/engine/tests/Functional.LTP/spec.json b/engine/tests/Functional.LTP/spec.json
index 3b48bdc..d23edc1 100644
--- a/engine/tests/Functional.LTP/spec.json
+++ b/engine/tests/Functional.LTP/spec.json
@@ -49,6 +49,7 @@
         },
         "quickhit": {
             "tests": "quickhit",
+            "noautoskip": "true",
             "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
             "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
         },
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 10/11] LTP: add test.yaml describing all variables
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (8 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 09/11] LTP: add noautoskip to the quickhit spec Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-18  2:48 ` [Fuego] [PATCH 11/11] LTP: add doc about fcntl35 Daniel Sangorrin
  2018-05-19  0:47 ` [Fuego] LTP: changes to the auto skip feature Tim.Bird
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

I moved some text written by Tim from fuego_test.sh, I hope
that is OK.
Also, I haven't properly validated the YAML file.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 engine/tests/Functional.LTP/fuego_test.sh | 23 ---------
 engine/tests/Functional.LTP/test.yaml     | 85 +++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+), 23 deletions(-)
 create mode 100644 engine/tests/Functional.LTP/test.yaml

diff --git a/engine/tests/Functional.LTP/fuego_test.sh b/engine/tests/Functional.LTP/fuego_test.sh
index 44296c5..ecbf758 100755
--- a/engine/tests/Functional.LTP/fuego_test.sh
+++ b/engine/tests/Functional.LTP/fuego_test.sh
@@ -57,29 +57,6 @@ func/sched_jitter
 #      - this is a one-time operation
 #      - this set up the system for 2
 #      - this is implemented by spec: "make_pkg"
-#
-# variables used:
-# from board file or spec:
-#   FUNCTIONAL_LTP_HOMEDIR = persistent location for LTP on board
-#   FUNCTIONAL_LTP_PHASES = list of phases to force doing
-#     - include string 'build' to do a build
-#     - include string 'deploy' to do a deploy
-#     - include string 'maketar' to do create a tarball for the target
-#     - include string 'run' to do a run
-#     - if empty, defaults to "build deploy run"
-#
-# scenario 1:
-#  FUNCTIONAL_LTP_HOMEDIR="", FUNTIONAL_LTP_PHASES=""
-# scenario 2:
-#  FUNCTIONAL_LTP_HOMEDIR="/opt/ltp", FUNCTIONAL_LTP_PHASES=""
-#  if not pre-installed, it's an error (detected in pre_check)
-# scenario 3:
-#  FUNCTIONAL_LTP_HOMEDIR="/opt/ltp", FUNCTIONAL_LTP_PHASES="build deploy"
-#  if already pre-installed, it's an error (or rm -rf is done in deploy)
-# scenario 4:
-#  FUNCTIONAL_LTP_HOMEDIR="/opt/ltp", FUNCTIONAL_LTP_PHASES="build maketar"
-#  user should copy tar to board, and install it, then set board file's
-#    FUNCTIONAL_LTP_HOMEDIR var
 
 # $1: string with tests to skip separated by spaces
 # $2: absolute path to the skipfile (default: ${LOGDIR}/skiplist.txt)
diff --git a/engine/tests/Functional.LTP/test.yaml b/engine/tests/Functional.LTP/test.yaml
new file mode 100644
index 0000000..714da45
--- /dev/null
+++ b/engine/tests/Functional.LTP/test.yaml
@@ -0,0 +1,85 @@
+fuego_package_version: 1
+name: Functional.LTP
+description: |
+    Linux Test Project is a joint project started by SGI, OSDL and
+    Bull developed and maintained by IBM, Cisco, Fujitsu, SUSE, Red Hat,
+    Oracle and others. The project goal is to deliver tests to the open
+    source community that validate the reliability, robustness, and
+    stability of Linux.
+license: GPL-2.0
+author: Andrew Tridgell, Ronnie Sahlberg
+maintainer: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
+version: 20180118
+fuego_release: 1
+type: Functional
+tags: ['kernel', 'syscalls', 'posix', 'real-time']
+tarball_src:
+    - ltp-full-20180118.tar.bz2: https://github.com/linux-test-project/ltp/releases/download/20180118/ltp-full-20180118.tar.bz2
+params:
+    - HOMEDIR:
+        description: persistent location for LTP on board
+        example: "/opt/ltp"
+        optional: yes
+    - PHASES:
+        description: |
+            - List of phases to force doing: |
+                - include string 'build' to do a build
+                - include string 'deploy' to do a deploy
+                - include string 'maketar' to do create a tarball for the target
+                - include string 'run' to do a run
+                - if empty, defaults to "build deploy run"
+            - Possible scenarios:
+                - scenario 1: |
+                    HOMEDIR="", PHASES=""
+                - scenario 2: |
+                    HOMEDIR="/opt/ltp", PHASES=""
+                    if not pre-installed, it is an error (detected in pre_check)
+                - scenario 3: |
+                    HOMEDIR="/opt/ltp", PHASES="build deploy"
+                    if already pre-installed, it is an error (or rm -rf is done in deploy)
+                - scenario 4: |
+                    HOMEDIR="/opt/ltp", PHASES="build maketar"
+                    user should copy tar to board, and install it, then set board file HOMEDIR var
+        example: "build maketar"
+        optional: yes
+    - NOAUTOSKIP:
+        description: |
+            Power users can use the NOAUTOSKIP flag to disable automatic
+            runtime skips. It has no effect on automatic build-time skips
+            (those done by skip_if_config_unavailable) unless you
+            rebuild LTP (Rebuild=true)
+        example: "true"
+        optional: yes
+    - TESTS:
+        description: |
+            The list of test groups (runtests) you want to run. You can choose
+            from the ALLTESTS variable inside fuego_test.sh
+        example: "syscalls fs pipes sched timers dio mm"
+        optional: no
+    - SKIPLIST:
+        description: |
+            List of test cases to skip or a path to a file containing a list
+            of test cases to skip.
+        example: "kill10 keyctl01 inotify06"
+        optional: yes
+    - BOARD_SKIPLIST:
+        description: Same as SKIPLIST but normally defined by the board file.
+        example: "abort01"
+        optional: yes
+data_files:
+    - chart_config.json
+    - criteria.json
+    - docs
+        - Functional.LTP.ftmp
+        - Functional.LTP.syscalls.add_key02.ftmp
+        - Functional.LTP.syscalls.fanotify07.ftmp
+        - Functional.LTP.syscalls.fcntl35.ftmp
+        - Functional.LTP.syscalls.ftmp
+        - Functional.LTP.syscalls.inotify06.ftmp
+    - fuego_test.sh
+    - ltp-full-20180118.tar.bz2
+    - ltp_process.py
+    - ltp_target_run.sh
+    - parser.py
+    - spec.json
+    - test.yaml
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Fuego] [PATCH 11/11] LTP: add doc about fcntl35
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (9 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 10/11] LTP: add test.yaml describing all variables Daniel Sangorrin
@ 2018-05-18  2:48 ` Daniel Sangorrin
  2018-05-19  0:47 ` [Fuego] LTP: changes to the auto skip feature Tim.Bird
  11 siblings, 0 replies; 13+ messages in thread
From: Daniel Sangorrin @ 2018-05-18  2:48 UTC (permalink / raw)
  To: fuego

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
 .../docs/Functional.LTP.syscalls.fcntl35.ftmp      | 53 ++++++++++++++++++++++
 1 file changed, 53 insertions(+)
 create mode 100644 engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.fcntl35.ftmp

diff --git a/engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.fcntl35.ftmp b/engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.fcntl35.ftmp
new file mode 100644
index 0000000..3a12f2e
--- /dev/null
+++ b/engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.fcntl35.ftmp
@@ -0,0 +1,53 @@
+===========
+Description
+===========
+Obtained from fcntl35.c DESCRIPTION:
+
+Description:
+fcntl(2) manpage states that an unprivileged user could not set the
+pipe capacity above the limit in /proc/sys/fs/pipe-max-size.  However,
+an unprivileged user could create a pipe whose initial capacity exceeds
+the limit.  We add a regression test to check that pipe-max-size caps
+the initial allocation for a new pipe for unprivileged users, but not
+for privileged users.
+
+This kernel bug has been fixed by:
+commit 086e774a57fba4695f14383c0818994c0b31da7c
+Author: Michael Kerrisk (man-pages) <mtk.manpages@gmail.com>
+Date:   Tue Oct 11 13:53:43 2016 -0700
+
+pipe: cap initial pipe capacity according to pipe-max-size limit
+
+====
+Tags
+====
+
+* kernel, syscall, pipe, fcntl
+
+=========
+Resources
+=========
+
+* http://git.emacinc.com/Linux-Kernel/linux-emac/commit/086e774a57fba4695f14383c0818994c0b31da7c?w=1
+
+=======
+Results
+=======
+
+Before porting commit 086e774a57fba4695f14383c0818994c0b31da7c:
+fcntl35.c:98: FAIL: an unprivileged user init the capacity of a pipe to 65536 unexpectedly, expected 4096
+
+After:
+fcntl35.c:101: PASS: an unprivileged user init the capacity of a pipe to 4096 successfully
+
+.. fuego_result_list::
+
+======
+Status
+======
+
+.. fuego_status::
+
+=====
+Notes
+=====
-- 
2.7.4



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [Fuego] LTP: changes to the auto skip feature
  2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
                   ` (10 preceding siblings ...)
  2018-05-18  2:48 ` [Fuego] [PATCH 11/11] LTP: add doc about fcntl35 Daniel Sangorrin
@ 2018-05-19  0:47 ` Tim.Bird
  11 siblings, 0 replies; 13+ messages in thread
From: Tim.Bird @ 2018-05-19  0:47 UTC (permalink / raw)
  To: daniel.sangorrin, fuego

I reviewed these, and they all look good to me.

I have applied them all.
 -- Tim

> -----Original Message-----
> From: Daniel Sangorrin
> I made a few changes to the auto skip feature that I added to LTP
> a while ago.
> 
> Patches:
> [PATCH 01/11] LTP: skip sbrk03 and others depending on architecture
> [PATCH 02/11] LTP: do not skip openat02
> [PATCH 03/11] LTP: fix small errata
> [PATCH 04/11] LTP: do not skip addkey02 and add a doc file
> [PATCH 05/11] LTP: do not skip adjtimex02
> [PATCH 06/11] LTP: add a kernel version check using version_lt
> [PATCH 07/11] LTP: skip fanotify07 depending on the kernel version
> [PATCH 08/11] LTP: add noautoskip option for power users
> [PATCH 09/11] LTP: add noautoskip to the quickhit spec
> [PATCH 10/11] LTP: add test.yaml describing all variables
> [PATCH 11/11] LTP: add doc about fcntl35
> 
> Stats:
>  engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.add_key02.ftmp |
> 62 ++++++++++++++++++++++++++++
>  engine/tests/Functional.LTP/docs/Functional.LTP.syscalls.fcntl35.ftmp   | 53
> ++++++++++++++++++++++++
>  engine/tests/Functional.LTP/fuego_test.sh                               | 89
> +++++++++++++++++++++++-----------------
>  engine/tests/Functional.LTP/spec.json                                   |  1 +
>  engine/tests/Functional.LTP/test.yaml                                   | 85
> ++++++++++++++++++++++++++++++++++++++
>  5 files changed, 253 insertions(+), 37 deletions(-)
> 
> Thanks,
> Daniel
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2018-05-19  0:47 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-18  2:48 [Fuego] LTP: changes to the auto skip feature Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 01/11] LTP: skip sbrk03 and others depending on architecture Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 02/11] LTP: do not skip openat02 Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 03/11] LTP: fix small errata Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 04/11] LTP: do not skip addkey02 and add a doc file Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 05/11] LTP: do not skip adjtimex02 Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 06/11] LTP: add a kernel version check using version_lt Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 07/11] LTP: skip fanotify07 depending on the kernel version Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 08/11] LTP: add noautoskip option for power users Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 09/11] LTP: add noautoskip to the quickhit spec Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 10/11] LTP: add test.yaml describing all variables Daniel Sangorrin
2018-05-18  2:48 ` [Fuego] [PATCH 11/11] LTP: add doc about fcntl35 Daniel Sangorrin
2018-05-19  0:47 ` [Fuego] LTP: changes to the auto skip feature Tim.Bird

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.