linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] selftests: Use kselftest skip code for skipped tests
@ 2021-05-25  6:17 Po-Hsu Lin
  2021-05-25 10:20 ` Petr Machata
  2021-05-25 13:56 ` Willem de Bruijn
  0 siblings, 2 replies; 4+ messages in thread
From: Po-Hsu Lin @ 2021-05-25  6:17 UTC (permalink / raw)
  To: linux-kselftest, netdev, bpf, linux-kernel
  Cc: po-hsu.lin, shuah, skhan, ast, daniel, andrii, kafai,
	songliubraving, yhs, john.fastabend, kpsingh, davem, kuba, hawk,
	nikolay, gnault, vladimir.oltean, idosch, baowen.zheng,
	danieller, petrm

There are several test cases still using exit 0 when they need to be
skipped. Use kselftest framework skip code instead so it can help us
to distinguish the proper return status.

Criterion to filter out what should be fixed in selftests directory:
  grep -r "exit 0" -B1 | grep -i skip

This change might cause some false-positives if people are running
these test scripts directly and only checking their return codes,
which will change from 0 to 4. However I think the impact should be
small as most of our scripts here are already using this skip code.
And there will be no such issue if running them with the kselftest
framework.

Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com>
---
 tools/testing/selftests/bpf/test_bpftool_build.sh             | 5 ++++-
 tools/testing/selftests/bpf/test_xdp_meta.sh                  | 5 ++++-
 tools/testing/selftests/bpf/test_xdp_vlan.sh                  | 7 +++++--
 tools/testing/selftests/net/fcnal-test.sh                     | 5 ++++-
 tools/testing/selftests/net/fib_rule_tests.sh                 | 7 +++++--
 tools/testing/selftests/net/forwarding/lib.sh                 | 5 ++++-
 tools/testing/selftests/net/forwarding/router_mpath_nh.sh     | 5 ++++-
 tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh | 5 ++++-
 tools/testing/selftests/net/run_afpackettests                 | 5 ++++-
 tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh        | 9 ++++++---
 tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh        | 9 ++++++---
 tools/testing/selftests/net/unicast_extensions.sh             | 5 ++++-
 tools/testing/selftests/net/vrf_strict_mode_test.sh           | 9 ++++++---
 tools/testing/selftests/ptp/phc.sh                            | 7 +++++--
 tools/testing/selftests/vm/charge_reserved_hugetlb.sh         | 5 ++++-
 tools/testing/selftests/vm/hugetlb_reparenting_test.sh        | 5 ++++-
 16 files changed, 73 insertions(+), 25 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_bpftool_build.sh b/tools/testing/selftests/bpf/test_bpftool_build.sh
index ac349a5..b6fab1e 100755
--- a/tools/testing/selftests/bpf/test_bpftool_build.sh
+++ b/tools/testing/selftests/bpf/test_bpftool_build.sh
@@ -1,6 +1,9 @@
 #!/bin/bash
 # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 case $1 in
 	-h|--help)
 		echo -e "$0 [-j <n>]"
@@ -22,7 +25,7 @@ KDIR_ROOT_DIR=$(realpath $PWD/$SCRIPT_REL_DIR/../../../../)
 cd $KDIR_ROOT_DIR
 if [ ! -e tools/bpf/bpftool/Makefile ]; then
 	echo -e "skip:    bpftool files not found!\n"
-	exit 0
+	exit $ksft_skip
 fi
 
 ERROR=0
diff --git a/tools/testing/selftests/bpf/test_xdp_meta.sh b/tools/testing/selftests/bpf/test_xdp_meta.sh
index 637fcf4..fd3f218 100755
--- a/tools/testing/selftests/bpf/test_xdp_meta.sh
+++ b/tools/testing/selftests/bpf/test_xdp_meta.sh
@@ -1,5 +1,8 @@
 #!/bin/sh
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 cleanup()
 {
 	if [ "$?" = "0" ]; then
@@ -17,7 +20,7 @@ cleanup()
 ip link set dev lo xdp off 2>/dev/null > /dev/null
 if [ $? -ne 0 ];then
 	echo "selftests: [SKIP] Could not run test without the ip xdp support"
-	exit 0
+	exit $ksft_skip
 fi
 set -e
 
diff --git a/tools/testing/selftests/bpf/test_xdp_vlan.sh b/tools/testing/selftests/bpf/test_xdp_vlan.sh
index bb8b0da..1aa7404 100755
--- a/tools/testing/selftests/bpf/test_xdp_vlan.sh
+++ b/tools/testing/selftests/bpf/test_xdp_vlan.sh
@@ -2,6 +2,9 @@
 # SPDX-License-Identifier: GPL-2.0
 # Author: Jesper Dangaard Brouer <hawk@kernel.org>
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 # Allow wrapper scripts to name test
 if [ -z "$TESTNAME" ]; then
     TESTNAME=xdp_vlan
@@ -94,7 +97,7 @@ while true; do
 	    -h | --help )
 		usage;
 		echo "selftests: $TESTNAME [SKIP] usage help info requested"
-		exit 0
+		exit $ksft_skip
 		;;
 	    * )
 		shift
@@ -117,7 +120,7 @@ fi
 ip link set dev lo xdpgeneric off 2>/dev/null > /dev/null
 if [ $? -ne 0 ]; then
 	echo "selftests: $TESTNAME [SKIP] need ip xdp support"
-	exit 0
+	exit $ksft_skip
 fi
 
 # Interactive mode likely require us to cleanup netns
diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
index a8ad928..9074e25 100755
--- a/tools/testing/selftests/net/fcnal-test.sh
+++ b/tools/testing/selftests/net/fcnal-test.sh
@@ -37,6 +37,9 @@
 #
 # server / client nomenclature relative to ns-A
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 VERBOSE=0
 
 NSA_DEV=eth1
@@ -3946,7 +3949,7 @@ fi
 which nettest >/dev/null
 if [ $? -ne 0 ]; then
 	echo "'nettest' command not found; skipping tests"
-	exit 0
+	exit $ksft_skip
 fi
 
 declare -i nfail=0
diff --git a/tools/testing/selftests/net/fib_rule_tests.sh b/tools/testing/selftests/net/fib_rule_tests.sh
index a93e6b6..43ea840 100755
--- a/tools/testing/selftests/net/fib_rule_tests.sh
+++ b/tools/testing/selftests/net/fib_rule_tests.sh
@@ -3,6 +3,9 @@
 
 # This test is for checking IPv4 and IPv6 FIB rules API
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 ret=0
 
 PAUSE_ON_FAIL=${PAUSE_ON_FAIL:=no}
@@ -238,12 +241,12 @@ run_fibrule_tests()
 
 if [ "$(id -u)" -ne 0 ];then
 	echo "SKIP: Need root privileges"
-	exit 0
+	exit $ksft_skip
 fi
 
 if [ ! -x "$(command -v ip)" ]; then
 	echo "SKIP: Could not run test without ip tool"
-	exit 0
+	exit $ksft_skip
 fi
 
 # start clean
diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
index 42e28c9..eed9f08 100644
--- a/tools/testing/selftests/net/forwarding/lib.sh
+++ b/tools/testing/selftests/net/forwarding/lib.sh
@@ -4,6 +4,9 @@
 ##############################################################################
 # Defines
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 # Can be overridden by the configuration file.
 PING=${PING:=ping}
 PING6=${PING6:=ping6}
@@ -121,7 +124,7 @@ check_ethtool_lanes_support()
 
 if [[ "$(id -u)" -ne 0 ]]; then
 	echo "SKIP: need root privileges"
-	exit 0
+	exit $ksft_skip
 fi
 
 if [[ "$CHECK_TC" = "yes" ]]; then
diff --git a/tools/testing/selftests/net/forwarding/router_mpath_nh.sh b/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
index 76efb1f..bb7dc6d 100755
--- a/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
+++ b/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
@@ -1,6 +1,9 @@
 #!/bin/bash
 # SPDX-License-Identifier: GPL-2.0
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 ALL_TESTS="
 	ping_ipv4
 	ping_ipv6
@@ -411,7 +414,7 @@ ping_ipv6()
 ip nexthop ls >/dev/null 2>&1
 if [ $? -ne 0 ]; then
 	echo "Nexthop objects not supported; skipping tests"
-	exit 0
+	exit $ksft_skip
 fi
 
 trap cleanup EXIT
diff --git a/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh b/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
index 4898dd4..e7bb976 100755
--- a/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
+++ b/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
@@ -1,6 +1,9 @@
 #!/bin/bash
 # SPDX-License-Identifier: GPL-2.0
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 ALL_TESTS="
 	ping_ipv4
 	ping_ipv6
@@ -386,7 +389,7 @@ ping_ipv6()
 ip nexthop ls >/dev/null 2>&1
 if [ $? -ne 0 ]; then
 	echo "Nexthop objects not supported; skipping tests"
-	exit 0
+	exit $ksft_skip
 fi
 
 trap cleanup EXIT
diff --git a/tools/testing/selftests/net/run_afpackettests b/tools/testing/selftests/net/run_afpackettests
index 8b42e8b..a59cb6a 100755
--- a/tools/testing/selftests/net/run_afpackettests
+++ b/tools/testing/selftests/net/run_afpackettests
@@ -1,9 +1,12 @@
 #!/bin/sh
 # SPDX-License-Identifier: GPL-2.0
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 if [ $(id -u) != 0 ]; then
 	echo $msg must be run as root >&2
-	exit 0
+	exit $ksft_skip
 fi
 
 ret=0
diff --git a/tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh b/tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh
index ad7a9fc..1003119 100755
--- a/tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh
+++ b/tools/testing/selftests/net/srv6_end_dt4_l3vpn_test.sh
@@ -163,6 +163,9 @@
 # +---------------------------------------------------+
 #
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 readonly LOCALSID_TABLE_ID=90
 readonly IPv6_RT_NETWORK=fd00
 readonly IPv4_HS_NETWORK=10.0.0
@@ -464,18 +467,18 @@ host_vpn_isolation_tests()
 
 if [ "$(id -u)" -ne 0 ];then
 	echo "SKIP: Need root privileges"
-	exit 0
+	exit $ksft_skip
 fi
 
 if [ ! -x "$(command -v ip)" ]; then
 	echo "SKIP: Could not run test without ip tool"
-	exit 0
+	exit $ksft_skip
 fi
 
 modprobe vrf &>/dev/null
 if [ ! -e /proc/sys/net/vrf/strict_mode ]; then
         echo "SKIP: vrf sysctl does not exist"
-        exit 0
+        exit $ksft_skip
 fi
 
 cleanup &>/dev/null
diff --git a/tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh b/tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh
index 68708f5..b9b06ef 100755
--- a/tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh
+++ b/tools/testing/selftests/net/srv6_end_dt6_l3vpn_test.sh
@@ -164,6 +164,9 @@
 # +---------------------------------------------------+
 #
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 readonly LOCALSID_TABLE_ID=90
 readonly IPv6_RT_NETWORK=fd00
 readonly IPv6_HS_NETWORK=cafe
@@ -472,18 +475,18 @@ host_vpn_isolation_tests()
 
 if [ "$(id -u)" -ne 0 ];then
 	echo "SKIP: Need root privileges"
-	exit 0
+	exit $ksft_skip
 fi
 
 if [ ! -x "$(command -v ip)" ]; then
 	echo "SKIP: Could not run test without ip tool"
-	exit 0
+	exit $ksft_skip
 fi
 
 modprobe vrf &>/dev/null
 if [ ! -e /proc/sys/net/vrf/strict_mode ]; then
         echo "SKIP: vrf sysctl does not exist"
-        exit 0
+        exit $ksft_skip
 fi
 
 cleanup &>/dev/null
diff --git a/tools/testing/selftests/net/unicast_extensions.sh b/tools/testing/selftests/net/unicast_extensions.sh
index dbf0421..728e4d5 100755
--- a/tools/testing/selftests/net/unicast_extensions.sh
+++ b/tools/testing/selftests/net/unicast_extensions.sh
@@ -28,12 +28,15 @@
 # These tests provide an easy way to flip the expected result of any
 # of these behaviors for testing kernel patches that change them.
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 # nettest can be run from PATH or from same directory as this selftest
 if ! which nettest >/dev/null; then
 	PATH=$PWD:$PATH
 	if ! which nettest >/dev/null; then
 		echo "'nettest' command not found; skipping tests"
-		exit 0
+		exit $ksft_skip
 	fi
 fi
 
diff --git a/tools/testing/selftests/net/vrf_strict_mode_test.sh b/tools/testing/selftests/net/vrf_strict_mode_test.sh
index 18b982d..865d53c 100755
--- a/tools/testing/selftests/net/vrf_strict_mode_test.sh
+++ b/tools/testing/selftests/net/vrf_strict_mode_test.sh
@@ -3,6 +3,9 @@
 
 # This test is designed for testing the new VRF strict_mode functionality.
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 ret=0
 
 # identifies the "init" network namespace which is often called root network
@@ -371,18 +374,18 @@ vrf_strict_mode_check_support()
 
 if [ "$(id -u)" -ne 0 ];then
 	echo "SKIP: Need root privileges"
-	exit 0
+	exit $ksft_skip
 fi
 
 if [ ! -x "$(command -v ip)" ]; then
 	echo "SKIP: Could not run test without ip tool"
-	exit 0
+	exit $ksft_skip
 fi
 
 modprobe vrf &>/dev/null
 if [ ! -e /proc/sys/net/vrf/strict_mode ]; then
 	echo "SKIP: vrf sysctl does not exist"
-	exit 0
+	exit $ksft_skip
 fi
 
 cleanup &> /dev/null
diff --git a/tools/testing/selftests/ptp/phc.sh b/tools/testing/selftests/ptp/phc.sh
index ac6e5a6..ca3c844c 100755
--- a/tools/testing/selftests/ptp/phc.sh
+++ b/tools/testing/selftests/ptp/phc.sh
@@ -1,6 +1,9 @@
 #!/bin/bash
 # SPDX-License-Identifier: GPL-2.0
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 ALL_TESTS="
 	settime
 	adjtime
@@ -13,12 +16,12 @@ DEV=$1
 
 if [[ "$(id -u)" -ne 0 ]]; then
 	echo "SKIP: need root privileges"
-	exit 0
+	exit $ksft_skip
 fi
 
 if [[ "$DEV" == "" ]]; then
 	echo "SKIP: PTP device not provided"
-	exit 0
+	exit $ksft_skip
 fi
 
 require_command()
diff --git a/tools/testing/selftests/vm/charge_reserved_hugetlb.sh b/tools/testing/selftests/vm/charge_reserved_hugetlb.sh
index 18d3368..fe8fcfb 100644
--- a/tools/testing/selftests/vm/charge_reserved_hugetlb.sh
+++ b/tools/testing/selftests/vm/charge_reserved_hugetlb.sh
@@ -1,11 +1,14 @@
 #!/bin/sh
 # SPDX-License-Identifier: GPL-2.0
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 set -e
 
 if [[ $(id -u) -ne 0 ]]; then
   echo "This test must be run as root. Skipping..."
-  exit 0
+  exit $ksft_skip
 fi
 
 fault_limit_file=limit_in_bytes
diff --git a/tools/testing/selftests/vm/hugetlb_reparenting_test.sh b/tools/testing/selftests/vm/hugetlb_reparenting_test.sh
index d11d1fe..4a9a3af 100644
--- a/tools/testing/selftests/vm/hugetlb_reparenting_test.sh
+++ b/tools/testing/selftests/vm/hugetlb_reparenting_test.sh
@@ -1,11 +1,14 @@
 #!/bin/bash
 # SPDX-License-Identifier: GPL-2.0
 
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
 set -e
 
 if [[ $(id -u) -ne 0 ]]; then
   echo "This test must be run as root. Skipping..."
-  exit 0
+  exit $ksft_skip
 fi
 
 usage_file=usage_in_bytes
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH] selftests: Use kselftest skip code for skipped tests
  2021-05-25  6:17 [PATCH] selftests: Use kselftest skip code for skipped tests Po-Hsu Lin
@ 2021-05-25 10:20 ` Petr Machata
  2021-05-25 10:43   ` Po-Hsu Lin
  2021-05-25 13:56 ` Willem de Bruijn
  1 sibling, 1 reply; 4+ messages in thread
From: Petr Machata @ 2021-05-25 10:20 UTC (permalink / raw)
  To: Po-Hsu Lin
  Cc: linux-kselftest, netdev, bpf, linux-kernel, shuah, skhan, ast,
	daniel, andrii, kafai, songliubraving, yhs, john.fastabend,
	kpsingh, davem, kuba, hawk, nikolay, gnault, vladimir.oltean,
	idosch, baowen.zheng, danieller, petrm


Po-Hsu Lin <po-hsu.lin@canonical.com> writes:

> diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
> index 42e28c9..eed9f08 100644
> --- a/tools/testing/selftests/net/forwarding/lib.sh
> +++ b/tools/testing/selftests/net/forwarding/lib.sh
> @@ -4,6 +4,9 @@
>  ##############################################################################
>  # Defines
>  
> +# Kselftest framework requirement - SKIP code is 4.
> +ksft_skip=4
> +
>  # Can be overridden by the configuration file.
>  PING=${PING:=ping}
>  PING6=${PING6:=ping6}
> @@ -121,7 +124,7 @@ check_ethtool_lanes_support()
>  
>  if [[ "$(id -u)" -ne 0 ]]; then
>  	echo "SKIP: need root privileges"
> -	exit 0
> +	exit $ksft_skip
>  fi
>  
>  if [[ "$CHECK_TC" = "yes" ]]; then
> diff --git a/tools/testing/selftests/net/forwarding/router_mpath_nh.sh b/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
> index 76efb1f..bb7dc6d 100755
> --- a/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
> +++ b/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
> @@ -1,6 +1,9 @@
>  #!/bin/bash
>  # SPDX-License-Identifier: GPL-2.0
>  
> +# Kselftest framework requirement - SKIP code is 4.
> +ksft_skip=4
> +
>  ALL_TESTS="
>  	ping_ipv4
>  	ping_ipv6
> @@ -411,7 +414,7 @@ ping_ipv6()
>  ip nexthop ls >/dev/null 2>&1
>  if [ $? -ne 0 ]; then
>  	echo "Nexthop objects not supported; skipping tests"
> -	exit 0
> +	exit $ksft_skip
>  fi
>  
>  trap cleanup EXIT

router_mpath_nh.sh sources lib.sh, which you changed above. This hunk
should not be necessary.

> diff --git a/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh b/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
> index 4898dd4..e7bb976 100755
> --- a/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
> +++ b/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
> @@ -1,6 +1,9 @@
>  #!/bin/bash
>  # SPDX-License-Identifier: GPL-2.0
>  
> +# Kselftest framework requirement - SKIP code is 4.
> +ksft_skip=4
> +
>  ALL_TESTS="
>  	ping_ipv4
>  	ping_ipv6
> @@ -386,7 +389,7 @@ ping_ipv6()
>  ip nexthop ls >/dev/null 2>&1
>  if [ $? -ne 0 ]; then
>  	echo "Nexthop objects not supported; skipping tests"
> -	exit 0
> +	exit $ksft_skip
>  fi
>  
>  trap cleanup EXIT

Likewise.

Unless I'm missing some indirect dependency, no other selftests in your
patch have this problem.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] selftests: Use kselftest skip code for skipped tests
  2021-05-25 10:20 ` Petr Machata
@ 2021-05-25 10:43   ` Po-Hsu Lin
  0 siblings, 0 replies; 4+ messages in thread
From: Po-Hsu Lin @ 2021-05-25 10:43 UTC (permalink / raw)
  To: Petr Machata
  Cc: linux-kselftest, netdev, bpf, linux-kernel, shuah, Shuah Khan,
	ast, daniel, andrii, kafai, songliubraving, yhs, john.fastabend,
	kpsingh, David Miller, Jakub Kicinski, hawk, nikolay, gnault,
	vladimir.oltean, idosch, baowen.zheng, danieller

On Tue, May 25, 2021 at 6:20 PM Petr Machata <petrm@nvidia.com> wrote:
>
>
> Po-Hsu Lin <po-hsu.lin@canonical.com> writes:
>
> > diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
> > index 42e28c9..eed9f08 100644
> > --- a/tools/testing/selftests/net/forwarding/lib.sh
> > +++ b/tools/testing/selftests/net/forwarding/lib.sh
> > @@ -4,6 +4,9 @@
> >  ##############################################################################
> >  # Defines
> >
> > +# Kselftest framework requirement - SKIP code is 4.
> > +ksft_skip=4
> > +
> >  # Can be overridden by the configuration file.
> >  PING=${PING:=ping}
> >  PING6=${PING6:=ping6}
> > @@ -121,7 +124,7 @@ check_ethtool_lanes_support()
> >
> >  if [[ "$(id -u)" -ne 0 ]]; then
> >       echo "SKIP: need root privileges"
> > -     exit 0
> > +     exit $ksft_skip
> >  fi
> >
> >  if [[ "$CHECK_TC" = "yes" ]]; then
> > diff --git a/tools/testing/selftests/net/forwarding/router_mpath_nh.sh b/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
> > index 76efb1f..bb7dc6d 100755
> > --- a/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
> > +++ b/tools/testing/selftests/net/forwarding/router_mpath_nh.sh
> > @@ -1,6 +1,9 @@
> >  #!/bin/bash
> >  # SPDX-License-Identifier: GPL-2.0
> >
> > +# Kselftest framework requirement - SKIP code is 4.
> > +ksft_skip=4
> > +
> >  ALL_TESTS="
> >       ping_ipv4
> >       ping_ipv6
> > @@ -411,7 +414,7 @@ ping_ipv6()
> >  ip nexthop ls >/dev/null 2>&1
> >  if [ $? -ne 0 ]; then
> >       echo "Nexthop objects not supported; skipping tests"
> > -     exit 0
> > +     exit $ksft_skip
> >  fi
> >
> >  trap cleanup EXIT
>
> router_mpath_nh.sh sources lib.sh, which you changed above. This hunk
> should not be necessary.

Hello Petr,
Nice catch, I will remove the following lines in those script that
sources lib.sh:
 +# Kselftest framework requirement - SKIP code is 4.
 +ksft_skip=4
 +

Will prepare V2 later, thanks!

>
> > diff --git a/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh b/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
> > index 4898dd4..e7bb976 100755
> > --- a/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
> > +++ b/tools/testing/selftests/net/forwarding/router_mpath_nh_res.sh
> > @@ -1,6 +1,9 @@
> >  #!/bin/bash
> >  # SPDX-License-Identifier: GPL-2.0
> >
> > +# Kselftest framework requirement - SKIP code is 4.
> > +ksft_skip=4
> > +
> >  ALL_TESTS="
> >       ping_ipv4
> >       ping_ipv6
> > @@ -386,7 +389,7 @@ ping_ipv6()
> >  ip nexthop ls >/dev/null 2>&1
> >  if [ $? -ne 0 ]; then
> >       echo "Nexthop objects not supported; skipping tests"
> > -     exit 0
> > +     exit $ksft_skip
> >  fi
> >
> >  trap cleanup EXIT
>
> Likewise.
>
> Unless I'm missing some indirect dependency, no other selftests in your
> patch have this problem.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] selftests: Use kselftest skip code for skipped tests
  2021-05-25  6:17 [PATCH] selftests: Use kselftest skip code for skipped tests Po-Hsu Lin
  2021-05-25 10:20 ` Petr Machata
@ 2021-05-25 13:56 ` Willem de Bruijn
  1 sibling, 0 replies; 4+ messages in thread
From: Willem de Bruijn @ 2021-05-25 13:56 UTC (permalink / raw)
  To: Po-Hsu Lin
  Cc: open list:KERNEL SELFTEST FRAMEWORK, Network Development, bpf,
	linux-kernel, Shuah Khan, skhan, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin Lau, Song Liu,
	Yonghong Song, John Fastabend, KP Singh, David Miller,
	Jakub Kicinski, Jesper Dangaard Brouer, nikolay, Guillaume Nault,
	Vladimir Oltean, idosch, baowen.zheng, danieller, petrm

On Tue, May 25, 2021 at 2:17 AM Po-Hsu Lin <po-hsu.lin@canonical.com> wrote:
>
> There are several test cases still using exit 0 when they need to be
> skipped. Use kselftest framework skip code instead so it can help us
> to distinguish the proper return status.
>
> Criterion to filter out what should be fixed in selftests directory:
>   grep -r "exit 0" -B1 | grep -i skip
>
> This change might cause some false-positives if people are running
> these test scripts directly and only checking their return codes,
> which will change from 0 to 4. However I think the impact should be
> small as most of our scripts here are already using this skip code.
> And there will be no such issue if running them with the kselftest
> framework.
>
> Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com>

Perhaps it's a good time to mention this patch to add a kselftest
helper library for shell tests again:

[PATCH] tools/testing: add kselftest shell helper library
https://lore.kernel.org/linux-kselftest/20201123162508.585279-1-willemdebruijn.kernel@gmail.com/

That can avoid redefining this constant in every file.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-05-25 14:04 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-25  6:17 [PATCH] selftests: Use kselftest skip code for skipped tests Po-Hsu Lin
2021-05-25 10:20 ` Petr Machata
2021-05-25 10:43   ` Po-Hsu Lin
2021-05-25 13:56 ` Willem de Bruijn

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).