* [PATCH v2 1/3] selftests: Extract run_kselftest.sh and generate stand-alone test list
2020-09-28 20:26 [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Kees Cook
@ 2020-09-28 20:26 ` Kees Cook
2020-10-01 5:09 ` Naresh Kamboju
2020-09-28 20:26 ` [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable Kees Cook
` (3 subsequent siblings)
4 siblings, 1 reply; 12+ messages in thread
From: Kees Cook @ 2020-09-28 20:26 UTC (permalink / raw)
To: Shuah Khan
Cc: Kees Cook, Naresh Kamboju, Hangbin Liu, Jonathan Corbet,
linux-doc, Tim.Bird, lkft-triage, Anders Roxell, Justin Cook,
Linux-Next Mailing List, linux-kernel, linux-kselftest
Instead of building a script on the fly (which just repeats the same
thing for each test collection), move the script out of the Makefile and
into run_kselftest.sh, which reads kselftest-list.txt.
Adjust the emit_tests target to report each test on a separate line so
that test running tools (e.g. LAVA) can easily remove individual
tests (for example, as seen in [1]).
[1] https://github.com/Linaro/test-definitions/pull/208/commits/2e7b62155e4998e54ac0587704932484d4ff84c8
Signed-off-by: Kees Cook <keescook@chromium.org>
---
tools/testing/selftests/Makefile | 26 ++++++----------------
tools/testing/selftests/lib.mk | 5 ++---
tools/testing/selftests/run_kselftest.sh | 28 ++++++++++++++++++++++++
3 files changed, 37 insertions(+), 22 deletions(-)
create mode 100755 tools/testing/selftests/run_kselftest.sh
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 15c1c1359c50..d9c283503159 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -206,6 +206,7 @@ KSFT_INSTALL_PATH := $(abspath $(KSFT_INSTALL_PATH))
# Avoid changing the rest of the logic here and lib.mk.
INSTALL_PATH := $(KSFT_INSTALL_PATH)
ALL_SCRIPT := $(INSTALL_PATH)/run_kselftest.sh
+TEST_LIST := $(INSTALL_PATH)/kselftest-list.txt
install: all
ifdef INSTALL_PATH
@@ -214,6 +215,8 @@ ifdef INSTALL_PATH
install -m 744 kselftest/module.sh $(INSTALL_PATH)/kselftest/
install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/
install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/
+ install -m 744 run_kselftest.sh $(INSTALL_PATH)/
+ rm -f $(TEST_LIST)
@ret=1; \
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
@@ -222,33 +225,18 @@ ifdef INSTALL_PATH
ret=$$((ret * $$?)); \
done; exit $$ret;
- @# Ask all targets to emit their test scripts
- echo "#!/bin/sh" > $(ALL_SCRIPT)
- echo "BASE_DIR=\$$(realpath \$$(dirname \$$0))" >> $(ALL_SCRIPT)
- echo "cd \$$BASE_DIR" >> $(ALL_SCRIPT)
- echo ". ./kselftest/runner.sh" >> $(ALL_SCRIPT)
- echo "ROOT=\$$PWD" >> $(ALL_SCRIPT)
- echo "if [ \"\$$1\" = \"--summary\" ]; then" >> $(ALL_SCRIPT)
- echo " logfile=\$$BASE_DIR/output.log" >> $(ALL_SCRIPT)
- echo " cat /dev/null > \$$logfile" >> $(ALL_SCRIPT)
- echo "fi" >> $(ALL_SCRIPT)
- @# While building run_kselftest.sh skip also non-existent TARGET dirs:
+ @# Ask all targets to emit their test scripts
+ @# While building kselftest-list.text skip also non-existent TARGET dirs:
@# they could be the result of a build failure and should NOT be
@# included in the generated runlist.
for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
[ ! -d $(INSTALL_PATH)/$$TARGET ] && echo "Skipping non-existent dir: $$TARGET" && continue; \
- echo "[ -w /dev/kmsg ] && echo \"kselftest: Running tests in $$TARGET\" >> /dev/kmsg" >> $(ALL_SCRIPT); \
- echo "cd $$TARGET" >> $(ALL_SCRIPT); \
- echo -n "run_many" >> $(ALL_SCRIPT); \
echo -n "Emit Tests for $$TARGET\n"; \
- $(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET -C $$TARGET emit_tests >> $(ALL_SCRIPT); \
- echo "" >> $(ALL_SCRIPT); \
- echo "cd \$$ROOT" >> $(ALL_SCRIPT); \
+ $(MAKE) -s --no-print-directory OUTPUT=$$BUILD_TARGET COLLECTION=$$TARGET \
+ -C $$TARGET emit_tests >> $(TEST_LIST); \
done;
-
- chmod u+x $(ALL_SCRIPT)
else
$(error Error: set INSTALL_PATH to use install)
endif
diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 51124b962d56..30848ca36555 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -107,9 +107,8 @@ endif
emit_tests:
for TEST in $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(TEST_PROGS); do \
BASENAME_TEST=`basename $$TEST`; \
- echo " \\"; \
- echo -n " \"$$BASENAME_TEST\""; \
- done; \
+ echo "$(COLLECTION):$$BASENAME_TEST"; \
+ done
# define if isn't already. It is undefined in make O= case.
ifeq ($(RM),)
diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh
new file mode 100755
index 000000000000..8b0ad4766d78
--- /dev/null
+++ b/tools/testing/selftests/run_kselftest.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# Run installed kselftest tests.
+#
+BASE_DIR=$(realpath $(dirname $0))
+cd $BASE_DIR
+TESTS="$BASE_DIR"/kselftest-list.txt
+if [ ! -r "$TESTS" ] ; then
+ echo "$0: Could not find list of tests to run ($TESTS)" >&2
+ exit 1
+fi
+available="$(cat "$TESTS")"
+
+. ./kselftest/runner.sh
+ROOT=$PWD
+
+if [ "$1" = "--summary" ] ; then
+ logfile="$BASE_DIR"/output.log
+ cat /dev/null > $logfile
+fi
+
+collections=$(echo "$available" | cut -d: -f1 | uniq)
+for collection in $collections ; do
+ [ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg
+ tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2)
+ (cd "$collection" && run_many $tests)
+done
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v2 1/3] selftests: Extract run_kselftest.sh and generate stand-alone test list
2020-09-28 20:26 ` [PATCH v2 1/3] selftests: " Kees Cook
@ 2020-10-01 5:09 ` Naresh Kamboju
0 siblings, 0 replies; 12+ messages in thread
From: Naresh Kamboju @ 2020-10-01 5:09 UTC (permalink / raw)
To: Kees Cook
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
>
> Instead of building a script on the fly (which just repeats the same
> thing for each test collection), move the script out of the Makefile and
> into run_kselftest.sh, which reads kselftest-list.txt.
>
> Adjust the emit_tests target to report each test on a separate line so
> that test running tools (e.g. LAVA) can easily remove individual
> tests (for example, as seen in [1]).
>
> [1] https://github.com/Linaro/test-definitions/pull/208/commits/2e7b62155e4998e54ac0587704932484d4ff84c8
>
> Signed-off-by: Kees Cook <keescook@chromium.org>
Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org>
- Naresh
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
2020-09-28 20:26 [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Kees Cook
2020-09-28 20:26 ` [PATCH v2 1/3] selftests: " Kees Cook
@ 2020-09-28 20:26 ` Kees Cook
2020-10-01 5:09 ` Naresh Kamboju
2020-10-15 9:27 ` Naresh Kamboju
2020-09-28 20:26 ` [PATCH v2 3/3] doc: dev-tools: kselftest.rst: Update examples and paths Kees Cook
` (2 subsequent siblings)
4 siblings, 2 replies; 12+ messages in thread
From: Kees Cook @ 2020-09-28 20:26 UTC (permalink / raw)
To: Shuah Khan
Cc: Kees Cook, Hangbin Liu, Naresh Kamboju, Jonathan Corbet,
linux-doc, Tim.Bird, lkft-triage, Anders Roxell, Justin Cook,
Linux-Next Mailing List, linux-kernel, linux-kselftest
Currently with run_kselftest.sh there is no way to choose which test
we could run. All the tests listed in kselftest-list.txt are all run
every time. This patch enhanced the run_kselftest.sh to make the test
collections (or tests) individually selectable. e.g.:
$ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
Additionally adds a way to list all known tests with "-l", usage
with "-h", and perform a dry run without running tests with "-n".
Co-developed-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
---
tools/testing/selftests/run_kselftest.sh | 77 ++++++++++++++++++++++--
1 file changed, 71 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh
index 8b0ad4766d78..609a4ef9300e 100755
--- a/tools/testing/selftests/run_kselftest.sh
+++ b/tools/testing/selftests/run_kselftest.sh
@@ -8,21 +8,86 @@ cd $BASE_DIR
TESTS="$BASE_DIR"/kselftest-list.txt
if [ ! -r "$TESTS" ] ; then
echo "$0: Could not find list of tests to run ($TESTS)" >&2
- exit 1
+ available=""
+else
+ available="$(cat "$TESTS")"
fi
-available="$(cat "$TESTS")"
. ./kselftest/runner.sh
ROOT=$PWD
-if [ "$1" = "--summary" ] ; then
- logfile="$BASE_DIR"/output.log
- cat /dev/null > $logfile
+usage()
+{
+ cat <<EOF
+Usage: $0 [OPTIONS]
+ -s | --summary Print summary with detailed log in output.log
+ -t | --test COLLECTION:TEST Run TEST from COLLECTION
+ -c | --collection COLLECTION Run all tests from COLLECTION
+ -l | --list List the available collection:test entries
+ -d | --dry-run Don't actually run any tests
+ -h | --help Show this usage info
+EOF
+ exit $1
+}
+
+COLLECTIONS=""
+TESTS=""
+dryrun=""
+while true; do
+ case "$1" in
+ -s | --summary)
+ logfile="$BASE_DIR"/output.log
+ cat /dev/null > $logfile
+ shift ;;
+ -t | --test)
+ TESTS="$TESTS $2"
+ shift 2 ;;
+ -c | --collection)
+ COLLECTIONS="$COLLECTIONS $2"
+ shift 2 ;;
+ -l | --list)
+ echo "$available"
+ exit 0 ;;
+ -n | --dry-run)
+ dryrun="echo"
+ shift ;;
+ -h | --help)
+ usage 0 ;;
+ "")
+ break ;;
+ *)
+ usage 1 ;;
+ esac
+done
+
+# Add all selected collections to the explicit test list.
+if [ -n "$COLLECTIONS" ]; then
+ for collection in $COLLECTIONS ; do
+ found="$(echo "$available" | grep "^$collection:")"
+ if [ -z "$found" ] ; then
+ echo "No such collection '$collection'" >&2
+ exit 1
+ fi
+ TESTS="$TESTS $found"
+ done
+fi
+# Replace available test list with explicitly selected tests.
+if [ -n "$TESTS" ]; then
+ valid=""
+ for test in $TESTS ; do
+ found="$(echo "$available" | grep "^${test}$")"
+ if [ -z "$found" ] ; then
+ echo "No such test '$test'" >&2
+ exit 1
+ fi
+ valid="$valid $found"
+ done
+ available="$(echo "$valid" | sed -e 's/ /\n/g')"
fi
collections=$(echo "$available" | cut -d: -f1 | uniq)
for collection in $collections ; do
[ -w /dev/kmsg ] && echo "kselftest: Running tests in $collection" >> /dev/kmsg
tests=$(echo "$available" | grep "^$collection:" | cut -d: -f2)
- (cd "$collection" && run_many $tests)
+ ($dryrun cd "$collection" && $dryrun run_many $tests)
done
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
2020-09-28 20:26 ` [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable Kees Cook
@ 2020-10-01 5:09 ` Naresh Kamboju
2020-10-15 9:27 ` Naresh Kamboju
1 sibling, 0 replies; 12+ messages in thread
From: Naresh Kamboju @ 2020-10-01 5:09 UTC (permalink / raw)
To: Kees Cook
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
>
> Currently with run_kselftest.sh there is no way to choose which test
> we could run. All the tests listed in kselftest-list.txt are all run
> every time. This patch enhanced the run_kselftest.sh to make the test
> collections (or tests) individually selectable. e.g.:
>
> $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
>
> Additionally adds a way to list all known tests with "-l", usage
> with "-h", and perform a dry run without running tests with "-n".
>
> Co-developed-by: Hangbin Liu <liuhangbin@gmail.com>
> Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
> Signed-off-by: Kees Cook <keescook@chromium.org>
Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org>
- Naresh
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
2020-09-28 20:26 ` [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable Kees Cook
2020-10-01 5:09 ` Naresh Kamboju
@ 2020-10-15 9:27 ` Naresh Kamboju
2020-10-15 21:57 ` Kees Cook
1 sibling, 1 reply; 12+ messages in thread
From: Naresh Kamboju @ 2020-10-15 9:27 UTC (permalink / raw)
To: Kees Cook
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
>
> Currently with run_kselftest.sh there is no way to choose which test
> we could run. All the tests listed in kselftest-list.txt are all run
> every time. This patch enhanced the run_kselftest.sh to make the test
> collections (or tests) individually selectable. e.g.:
>
> $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
>
> Additionally adds a way to list all known tests with "-l", usage
> with "-h", and perform a dry run without running tests with "-n".
While testing this patch set on LAVA the skip test functionality is not working.
We may have to revisit test definitions kselftest skip logic
or else
may add one more option to skip a given test on run_kselftest.sh script.
ref:
https://github.com/Linaro/test-definitions/blob/master/automated/linux/kselftest/kselftest.sh#L196
- Naresh
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable
2020-10-15 9:27 ` Naresh Kamboju
@ 2020-10-15 21:57 ` Kees Cook
0 siblings, 0 replies; 12+ messages in thread
From: Kees Cook @ 2020-10-15 21:57 UTC (permalink / raw)
To: Naresh Kamboju
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Thu, Oct 15, 2020 at 02:57:34PM +0530, Naresh Kamboju wrote:
> On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
> >
> > Currently with run_kselftest.sh there is no way to choose which test
> > we could run. All the tests listed in kselftest-list.txt are all run
> > every time. This patch enhanced the run_kselftest.sh to make the test
> > collections (or tests) individually selectable. e.g.:
> >
> > $ ./run_kselftest.sh -c seccomp -t timers:posix_timers -t timers:nanosleep
> >
> > Additionally adds a way to list all known tests with "-l", usage
> > with "-h", and perform a dry run without running tests with "-n".
>
>
> While testing this patch set on LAVA the skip test functionality is not working.
> We may have to revisit test definitions kselftest skip logic
> or else
> may add one more option to skip a given test on run_kselftest.sh script.
>
> ref:
> https://github.com/Linaro/test-definitions/blob/master/automated/linux/kselftest/kselftest.sh#L196
Yes, LAVA's hack to skip tests needs to be adjusted. Here's what it
should probably look like:
https://github.com/Linaro/test-definitions/pull/231
--
Kees Cook
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2 3/3] doc: dev-tools: kselftest.rst: Update examples and paths
2020-09-28 20:26 [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Kees Cook
2020-09-28 20:26 ` [PATCH v2 1/3] selftests: " Kees Cook
2020-09-28 20:26 ` [PATCH v2 2/3] selftests/run_kselftest.sh: Make each test individually selectable Kees Cook
@ 2020-09-28 20:26 ` Kees Cook
2020-10-01 5:13 ` Naresh Kamboju
2020-09-29 1:53 ` [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Hangbin Liu
2020-09-30 16:17 ` Naresh Kamboju
4 siblings, 1 reply; 12+ messages in thread
From: Kees Cook @ 2020-09-28 20:26 UTC (permalink / raw)
To: Shuah Khan
Cc: Kees Cook, Naresh Kamboju, Hangbin Liu, Jonathan Corbet,
linux-doc, Tim.Bird, lkft-triage, Anders Roxell, Justin Cook,
Linux-Next Mailing List, linux-kernel, linux-kselftest
Update the installation commands and path details, detail the new
options available in the run_kselftests.sh script.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
Documentation/dev-tools/kselftest.rst | 35 +++++++++++++++++----------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
index 469d115a95f1..a901def730d9 100644
--- a/Documentation/dev-tools/kselftest.rst
+++ b/Documentation/dev-tools/kselftest.rst
@@ -125,32 +125,41 @@ Note that some tests will require root privileges.
Install selftests
=================
-You can use the kselftest_install.sh tool to install selftests in the
-default location, which is tools/testing/selftests/kselftest, or in a
-user specified location.
+You can use the "install" target of "make" (which calls the `kselftest_install.sh`
+tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
+or in a user specified location via the `INSTALL_PATH` "make" variable.
To install selftests in default location::
- $ cd tools/testing/selftests
- $ ./kselftest_install.sh
+ $ make -C tools/testing/selftests install
To install selftests in a user specified location::
- $ cd tools/testing/selftests
- $ ./kselftest_install.sh install_dir
+ $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
Running installed selftests
===========================
-Kselftest install as well as the Kselftest tarball provide a script
-named "run_kselftest.sh" to run the tests.
+Found in the install directory, as well as in the Kselftest tarball,
+is a script named `run_kselftest.sh` to run the tests.
You can simply do the following to run the installed Kselftests. Please
note some tests will require root privileges::
- $ cd kselftest
+ $ cd kselftest_install
$ ./run_kselftest.sh
+To see the list of available tests, the `-l` option can be used::
+
+ $ ./run_kselftest.sh -l
+
+The `-c` option can be used to run all the tests from a test collection, or
+the `-t` option for specific single tests. Either can be used multiple times::
+
+ $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
+
+For other features see the script usage output, seen with the `-h` option.
+
Packaging selftests
===================
@@ -160,9 +169,9 @@ different system. To package selftests, run::
$ make -C tools/testing/selftests gen_tar
This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
-default, `.gz` format is used. The tar format can be overridden by specifying
-a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ option
-is supported, such as::
+default, `.gz` format is used. The tar compression format can be overridden by
+specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
+option is supported, such as::
$ make -C tools/testing/selftests gen_tar FORMAT=.xz
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v2 3/3] doc: dev-tools: kselftest.rst: Update examples and paths
2020-09-28 20:26 ` [PATCH v2 3/3] doc: dev-tools: kselftest.rst: Update examples and paths Kees Cook
@ 2020-10-01 5:13 ` Naresh Kamboju
0 siblings, 0 replies; 12+ messages in thread
From: Naresh Kamboju @ 2020-10-01 5:13 UTC (permalink / raw)
To: Kees Cook
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
>
> Update the installation commands and path details, detail the new
> options available in the run_kselftests.sh script.
>
> Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Naresh Kamboju <naresh.kamboju@linaro.org>
> ---
> Documentation/dev-tools/kselftest.rst | 35 +++++++++++++++++----------
> 1 file changed, 22 insertions(+), 13 deletions(-)
>
> diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
> index 469d115a95f1..a901def730d9 100644
> --- a/Documentation/dev-tools/kselftest.rst
> +++ b/Documentation/dev-tools/kselftest.rst
> @@ -125,32 +125,41 @@ Note that some tests will require root privileges.
> Install selftests
> =================
>
> -You can use the kselftest_install.sh tool to install selftests in the
> -default location, which is tools/testing/selftests/kselftest, or in a
> -user specified location.
> +You can use the "install" target of "make" (which calls the `kselftest_install.sh`
> +tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
> +or in a user specified location via the `INSTALL_PATH` "make" variable.
>
> To install selftests in default location::
>
> - $ cd tools/testing/selftests
> - $ ./kselftest_install.sh
> + $ make -C tools/testing/selftests install
>
> To install selftests in a user specified location::
>
> - $ cd tools/testing/selftests
> - $ ./kselftest_install.sh install_dir
> + $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
>
> Running installed selftests
> ===========================
>
> -Kselftest install as well as the Kselftest tarball provide a script
> -named "run_kselftest.sh" to run the tests.
> +Found in the install directory, as well as in the Kselftest tarball,
> +is a script named `run_kselftest.sh` to run the tests.
>
> You can simply do the following to run the installed Kselftests. Please
> note some tests will require root privileges::
>
> - $ cd kselftest
> + $ cd kselftest_install
> $ ./run_kselftest.sh
>
> +To see the list of available tests, the `-l` option can be used::
> +
> + $ ./run_kselftest.sh -l
> +
> +The `-c` option can be used to run all the tests from a test collection, or
> +the `-t` option for specific single tests. Either can be used multiple times::
> +
> + $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
> +
> +For other features see the script usage output, seen with the `-h` option.
> +
> Packaging selftests
> ===================
>
> @@ -160,9 +169,9 @@ different system. To package selftests, run::
> $ make -C tools/testing/selftests gen_tar
>
> This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
> -default, `.gz` format is used. The tar format can be overridden by specifying
> -a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ option
> -is supported, such as::
> +default, `.gz` format is used. The tar compression format can be overridden by
> +specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
> +option is supported, such as::
>
> $ make -C tools/testing/selftests gen_tar FORMAT=.xz
>
> --
> 2.25.1
>
- Naresh
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list
2020-09-28 20:26 [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Kees Cook
` (2 preceding siblings ...)
2020-09-28 20:26 ` [PATCH v2 3/3] doc: dev-tools: kselftest.rst: Update examples and paths Kees Cook
@ 2020-09-29 1:53 ` Hangbin Liu
2020-09-30 16:17 ` Naresh Kamboju
4 siblings, 0 replies; 12+ messages in thread
From: Hangbin Liu @ 2020-09-29 1:53 UTC (permalink / raw)
To: Kees Cook
Cc: Shuah Khan, Naresh Kamboju, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
linux-kernel, linux-kselftest
On Mon, Sep 28, 2020 at 01:26:47PM -0700, Kees Cook wrote:
> v2:
> - update documentation
> - include SPDX line in extracted script
> v1: https://lore.kernel.org/linux-kselftest/20200925234527.1885234-1-keescook@chromium.org/
>
I'm not sure if the doc update are all appropriate. Need others help review.
The script part looks good to me. Thanks for your update.
Regards
Hangbin
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list
2020-09-28 20:26 [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Kees Cook
` (3 preceding siblings ...)
2020-09-29 1:53 ` [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list Hangbin Liu
@ 2020-09-30 16:17 ` Naresh Kamboju
2020-09-30 23:25 ` Kees Cook
4 siblings, 1 reply; 12+ messages in thread
From: Naresh Kamboju @ 2020-09-30 16:17 UTC (permalink / raw)
To: Kees Cook
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
>
> v2:
> - update documentation
> - include SPDX line in extracted script
> v1: https://lore.kernel.org/linux-kselftest/20200925234527.1885234-1-keescook@chromium.org/
>
>
> Hi!
>
> I really like Hangbin Liu's intent[1] but I think we need to be a little
> more clean about the implementation. This extracts run_kselftest.sh from
> the Makefile so it can actually be changed without embeds, etc. Instead,
> generate the test list into a text file. Everything gets much simpler.
> :)
>
> And in patch 2, I add back Hangbin Liu's new options (with some extra
> added) with knowledge of "collections" (i.e. Makefile TARGETS) and
> subtests. This should work really well with LAVA too, which needs to
> manipulate the lists of tests being run.
>
> Thoughts?
I have tested this patch set on LAVA with full run and it went well.
>
> -Kees
>
> [1] https://lore.kernel.org/lkml/20200914022227.437143-1-liuhangbin@gmail.com/
>
> Kees Cook (3):
> selftests: Extract run_kselftest.sh and generate stand-alone test list
> selftests/run_kselftest.sh: Make each test individually selectable
> doc: dev-tools: kselftest.rst: Update examples and paths
>
> Documentation/dev-tools/kselftest.rst | 35 +++++----
> tools/testing/selftests/Makefile | 26 ++-----
> tools/testing/selftests/lib.mk | 5 +-
> tools/testing/selftests/run_kselftest.sh | 93 ++++++++++++++++++++++++
> 4 files changed, 124 insertions(+), 35 deletions(-)
> create mode 100755 tools/testing/selftests/run_kselftest.sh
>
> --
> 2.25.1
>
- Naresh
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v2 0/3] Extract run_kselftest.sh and generate stand-alone test list
2020-09-30 16:17 ` Naresh Kamboju
@ 2020-09-30 23:25 ` Kees Cook
0 siblings, 0 replies; 12+ messages in thread
From: Kees Cook @ 2020-09-30 23:25 UTC (permalink / raw)
To: Naresh Kamboju
Cc: Shuah Khan, Hangbin Liu, Jonathan Corbet, linux-doc, Tim.Bird,
lkft-triage, Anders Roxell, Justin Cook, Linux-Next Mailing List,
open list, open list:KERNEL SELFTEST FRAMEWORK
On Wed, Sep 30, 2020 at 09:47:49PM +0530, Naresh Kamboju wrote:
> On Tue, 29 Sep 2020 at 01:56, Kees Cook <keescook@chromium.org> wrote:
> >
> > v2:
> > - update documentation
> > - include SPDX line in extracted script
> > v1: https://lore.kernel.org/linux-kselftest/20200925234527.1885234-1-keescook@chromium.org/
> >
> >
> > Hi!
> >
> > I really like Hangbin Liu's intent[1] but I think we need to be a little
> > more clean about the implementation. This extracts run_kselftest.sh from
> > the Makefile so it can actually be changed without embeds, etc. Instead,
> > generate the test list into a text file. Everything gets much simpler.
> > :)
> >
> > And in patch 2, I add back Hangbin Liu's new options (with some extra
> > added) with knowledge of "collections" (i.e. Makefile TARGETS) and
> > subtests. This should work really well with LAVA too, which needs to
> > manipulate the lists of tests being run.
> >
> > Thoughts?
>
> I have tested this patch set on LAVA with full run and it went well.
Thank you! You can include this as a tag too, so a "b4 am" will pick it
up:
Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org>
-Kees
>
> >
> > -Kees
> >
> > [1] https://lore.kernel.org/lkml/20200914022227.437143-1-liuhangbin@gmail.com/
> >
> > Kees Cook (3):
> > selftests: Extract run_kselftest.sh and generate stand-alone test list
> > selftests/run_kselftest.sh: Make each test individually selectable
> > doc: dev-tools: kselftest.rst: Update examples and paths
> >
> > Documentation/dev-tools/kselftest.rst | 35 +++++----
> > tools/testing/selftests/Makefile | 26 ++-----
> > tools/testing/selftests/lib.mk | 5 +-
> > tools/testing/selftests/run_kselftest.sh | 93 ++++++++++++++++++++++++
> > 4 files changed, 124 insertions(+), 35 deletions(-)
> > create mode 100755 tools/testing/selftests/run_kselftest.sh
> >
> > --
> > 2.25.1
> >
>
> - Naresh
--
Kees Cook
^ permalink raw reply [flat|nested] 12+ messages in thread