[PATCH 00/20] kselftest install target feature

[PATCH 00/20] kselftest install target feature

[Shuah Khan]

This patch series adds a new kselftest_install make target
to enable selftest install. When make kselftest_install is
run, selftests are installed on the system. A new install
target is added to selftests Makefile which will install
targets for the tests that are specified in INSTALL_TARGETS.
During install, a script is generated to run tests that are
installed. This script will be installed in the selftest install
directory. Individual test Makefiles are changed to add to the
script. This will allow new tests to add install and run test
commands to the generated kselftest script.

Approach:

make kselftest_target:
-- exports kselftest INSTALL_KSFT_PATH
   default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
-- exports path for ksefltest.sh
-- runs selftests make install target:

selftests make install target
-- creates kselftest.sh script in install install dir
-- runs install targets for all INSTALL_TARGETS

Individual test make install targets:
-- install test programs and/or scripts in install dir
-- append to the ksefltest.sh file to add commands to run test

Shuah Khan (20):
  selftests/user: move test out of Makefile into a shell script
  selftests/net: move test out of Makefile into a shell script
  kbuild: kselftest_install - add a new make target to install selftests
  selftests: add install target to enable installing selftests
  selftests/breakpoints: add install target to enable installing test
  selftests/cpu-hotplug: add install target to enable installing test
  selftests/efivarfs: add install target to enable installing test
  selftests/firmware: add install target to enable installing test
  selftests/ipc: add install target to enable installing test
  selftests/kcmp: add install target to enable installing test
  selftests/memfd: add install target to enable installing test
  selftests/memory-hotplug: add install target to enable installing test
  selftests/mount: add install target to enable installing test
  selftests/mqueue: add install target to enable installing test
  selftests/net: add install target to enable installing test
  selftests/ptrace: add install target to enable installing test
  selftests/sysctl: add install target to enable installing test
  selftests/timers: add install target to enable installing test
  selftests/vm: add install target to enable installing test
  selftests/user: add install target to enable installing test

 Makefile                                        | 17 +++++++++++++++++
 tools/testing/selftests/Makefile                | 14 ++++++++++++++
 tools/testing/selftests/breakpoints/Makefile    | 12 ++++++++++++
 tools/testing/selftests/cpu-hotplug/Makefile    |  9 +++++++++
 tools/testing/selftests/efivarfs/Makefile       | 13 ++++++++++++-
 tools/testing/selftests/firmware/Makefile       | 20 ++++++++++++++++++++
 tools/testing/selftests/ipc/Makefile            | 11 +++++++++++
 tools/testing/selftests/kcmp/Makefile           | 12 ++++++++++++
 tools/testing/selftests/memfd/Makefile          | 10 ++++++++++
 tools/testing/selftests/memory-hotplug/Makefile |  9 +++++++++
 tools/testing/selftests/mount/Makefile          |  7 +++++++
 tools/testing/selftests/mqueue/Makefile         |  8 ++++++++
 tools/testing/selftests/net/Makefile            | 18 +++++++++++-------
 tools/testing/selftests/net/test_bpf.sh         | 10 ++++++++++
 tools/testing/selftests/ptrace/Makefile         | 11 +++++++++--
 tools/testing/selftests/sysctl/Makefile         | 10 ++++++++++
 tools/testing/selftests/timers/Makefile         |  7 +++++++
 tools/testing/selftests/user/Makefile           | 15 ++++++++-------
 tools/testing/selftests/user/test_user_copy.sh  | 10 ++++++++++
 tools/testing/selftests/vm/Makefile             |  7 +++++++
 20 files changed, 213 insertions(+), 17 deletions(-)
 create mode 100755 tools/testing/selftests/net/test_bpf.sh
 create mode 100755 tools/testing/selftests/user/test_user_copy.sh

-- 
1.9.1

[PATCH 01/20] selftests/user: move test out of Makefile into a shell script

[Shuah Khan]

Currently user copy test is run from the Makefile. Move it out
of the Makefile to be run from a shell script to allow the test
to be run as stand-alone test, in addition to allowing the test
run from a make target.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/user/Makefile          |  8 +-------
 tools/testing/selftests/user/test_user_copy.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 7 deletions(-)
 create mode 100755 tools/testing/selftests/user/test_user_copy.sh

diff --git a/tools/testing/selftests/user/Makefile b/tools/testing/selftests/user/Makefile
index 396255b..12c9d15 100644
--- a/tools/testing/selftests/user/Makefile
+++ b/tools/testing/selftests/user/Makefile
@@ -4,10 +4,4 @@
 all:
 
 run_tests: all
-	@if /sbin/modprobe test_user_copy ; then \
-		rmmod test_user_copy; \
-		echo "user_copy: ok"; \
-	else \
-		echo "user_copy: [FAIL]"; \
-		exit 1; \
-	fi
+	./test_user_copy.sh
diff --git a/tools/testing/selftests/user/test_user_copy.sh b/tools/testing/selftests/user/test_user_copy.sh
new file mode 100755
index 0000000..350107f
--- /dev/null
+++ b/tools/testing/selftests/user/test_user_copy.sh
@@ -0,0 +1,10 @@
+#!/bin/sh
+# Runs copy_to/from_user infrastructure using test_user_copy kernel module
+
+if /sbin/modprobe -q test_user_copy; then
+	/sbin/modprobe -q -r test_user_copy
+	echo "user_copy: ok"
+else
+	echo "user_copy: [FAIL]"
+	exit 1
+fi
-- 
1.9.1

[PATCH 02/20] selftests/net: move test out of Makefile into a shell script

[Shuah Khan]

Currently bpf test run from the Makefile. Move it out of the
Makefile to be run from a shell script to allow the test to
be run as stand-alone test, in addition to allowing the test
run from a make target.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/net/Makefile    |  8 +-------
 tools/testing/selftests/net/test_bpf.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 7 deletions(-)
 create mode 100755 tools/testing/selftests/net/test_bpf.sh

diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index c7493b8..62f22cc 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -14,12 +14,6 @@ all: $(NET_PROGS)
 run_tests: all
 	@/bin/sh ./run_netsocktests || echo "sockettests: [FAIL]"
 	@/bin/sh ./run_afpackettests || echo "afpackettests: [FAIL]"
-	@if /sbin/modprobe test_bpf ; then \
-		/sbin/rmmod test_bpf; \
-		echo "test_bpf: ok"; \
-	else \
-		echo "test_bpf: [FAIL]"; \
-		exit 1; \
-	fi
+	./test_bpf.sh
 clean:
 	$(RM) $(NET_PROGS)
diff --git a/tools/testing/selftests/net/test_bpf.sh b/tools/testing/selftests/net/test_bpf.sh
new file mode 100755
index 0000000..8b29796
--- /dev/null
+++ b/tools/testing/selftests/net/test_bpf.sh
@@ -0,0 +1,10 @@
+#!/bin/sh
+# Runs bpf test using test_bpf kernel module
+
+if /sbin/modprobe -q test_bpf ; then
+	/sbin/modprobe -q -r test_bpf;
+	echo "test_bpf: ok";
+else
+	echo "test_bpf: [FAIL]";
+	exit 1;
+fi
-- 
1.9.1

[PATCH 03/20] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

Add a new make target to install to install kernel selftests.
This new target will build and install selftests.

Approach:

make kselftest_target:
-- exports kselftest INSTALL_KSFT_PATH
   default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
-- exports path for ksefltest.sh
-- runs selftests make install target:

selftests make install target
-- creates kselftest.sh script in install install dir
-- runs install targets for all INSTALL_TARGETS

Individual test make install targets:
-- install test programs and/or scripts in install dir
-- append to the ksefltest.sh file to add commands to run test

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 Makefile | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/Makefile b/Makefile
index 05d67af..291aff7 100644
--- a/Makefile
+++ b/Makefile
@@ -1078,6 +1078,20 @@ kselftest:
 	$(Q)$(MAKE) -C tools/testing/selftests run_tests
 
 # ---------------------------------------------------------------------------
+# Kernel selftest install
+INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
+export INSTALL_KSFT_PATH
+KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
+export KSELFTEST
+
+PHONY += kselftest_install
+kselftest_install:
+	@rm -rf $(INSTALL_KSFT_PATH)
+	@mkdir -p $(INSTALL_KSFT_PATH)
+	$(Q)$(MAKE) -C tools/testing/selftests install
+	chmod +x $(KSELFTEST)
+
+# ---------------------------------------------------------------------------
 # Modules
 
 ifdef CONFIG_MODULES
@@ -1285,6 +1299,9 @@ help:
 	@echo  '                    Build, install, and boot kernel before'
 	@echo  '                    running kselftest on it'
 	@echo  ''
+	@echo  '  kselftest_install - Install Kselftests to INSTALL_KSFT_PATH'
+	@echo  '                      default: $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)'
+	@echo  ''
 	@echo  'Kernel packaging:'
 	@$(MAKE) $(build)=$(package-dir) help
 	@echo  ''
-- 
1.9.1

[PATCH 04/20] selftests: add install target to enable installing selftests

[Shuah Khan]

Add a new make target to enable installing selftests. This
new target will call install targets for the tests that are
specified in INSTALL_TARGETS. During install, a script is
generated to run tests that are installed. This script will
be installed in the selftest install directory. Individual
test Makefiles are changed to add to the script. This will
allow new tests to add install and run test commands to the
generated kselftest script.

Approach:

make kselftest_target:
-- exports kselftest INSTALL_KSFT_PATH
   default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
-- exports path for ksefltest.sh
-- runs selftests make install target:

selftests make install target
-- creates kselftest.sh script in install install dir
-- runs install targets for all INSTALL_TARGETS

Individual test make install targets:
-- install test programs and/or scripts in install dir
-- append to the ksefltest.sh file to add commands to run test

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/Makefile | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 45f145c..07b0244 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -16,6 +16,10 @@ TARGETS += sysctl
 TARGETS += firmware
 TARGETS += ftrace
 
+INSTALL_TARGETS = breakpoints cpu-hotplug efivarfs firmware ipc
+INSTALL_TARGETS += kcmp memfd memory-hotplug mqueue mount net
+INSTALL_TARGETS += ptrace sysctl timers user vm
+
 TARGETS_HOTPLUG = cpu-hotplug
 TARGETS_HOTPLUG += memory-hotplug
 
@@ -24,6 +28,16 @@ all:
 		make -C $$TARGET; \
 	done;
 
+install: all
+	echo "#!/bin/sh\n# Kselftest Run Tests ...." > $(KSELFTEST)
+	echo "# This file is generated during kselftest_install" >> $(KSELFTEST)
+	echo "# Please don't change it !!\n"  >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+	for TARGET in $(INSTALL_TARGETS); do \
+		echo "\nInstalling $$TARGET"; \
+		make -C $$TARGET install; \
+	done;
+
 run_tests: all
 	for TARGET in $(TARGETS); do \
 		make -C $$TARGET run_tests; \
-- 
1.9.1

[PATCH 05/20] selftests/breakpoints: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/breakpoints/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tools/testing/selftests/breakpoints/Makefile b/tools/testing/selftests/breakpoints/Makefile
index e18b42b..4b4e385 100644
--- a/tools/testing/selftests/breakpoints/Makefile
+++ b/tools/testing/selftests/breakpoints/Makefile
@@ -8,6 +8,7 @@ ifeq ($(ARCH),x86_64)
 	ARCH := x86
 endif
 
+TEST_STR = "./breakpoint_test || echo \"breakpoints selftests: [FAIL]\""
 
 all:
 ifeq ($(ARCH),x86)
@@ -16,6 +17,17 @@ else
 	echo "Not an x86 target, can't build breakpoints selftests"
 endif
 
+install: all
+ifeq ($(ARCH),x86)
+	install ./breakpoint_test $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start breakpoints test ....\"" >> $(KSELFTEST)
+	echo $(TEST_STR) >> $(KSELFTEST)
+	echo "echo \"End breakpoints test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+else
+	echo "Not an x86 target, can't install breakpoints selftests"
+endif
+
 run_tests:
 	@./breakpoint_test || echo "breakpoints selftests: [FAIL]"
 
-- 
1.9.1

[PATCH 06/20] selftests/cpu-hotplug: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/cpu-hotplug/Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/testing/selftests/cpu-hotplug/Makefile b/tools/testing/selftests/cpu-hotplug/Makefile
index e9c28d8..3075d3a 100644
--- a/tools/testing/selftests/cpu-hotplug/Makefile
+++ b/tools/testing/selftests/cpu-hotplug/Makefile
@@ -1,5 +1,14 @@
+TEST_STR="/bin/bash ./cpu-on-off-test.sh || echo \"cpu-hotplug selftests: [FAIL]\""
+
 all:
 
+install:
+	install ./on-off-test.sh $(INSTALL_KSFT_PATH)/cpu-on-off-test.sh
+	echo "\necho \"Start cpu hotplug test ....\"" >> $(KSELFTEST)
+	echo $(TEST_STR) >> $(KSELFTEST)
+	echo "echo \"End cpu hotplug test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests:
 	@/bin/bash ./on-off-test.sh || echo "cpu-hotplug selftests: [FAIL]"
 
-- 
1.9.1

[PATCH 07/20] selftests/efivarfs: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/efivarfs/Makefile | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/efivarfs/Makefile b/tools/testing/selftests/efivarfs/Makefile
index 29e8c6b..a668728 100644
--- a/tools/testing/selftests/efivarfs/Makefile
+++ b/tools/testing/selftests/efivarfs/Makefile
@@ -3,7 +3,18 @@ CFLAGS = -Wall
 
 test_objs = open-unlink create-read
 
-all: $(test_objs)
+TEST_STR = "/bin/bash ./efivarfs.sh || echo \"efivarfs selftests: [FAIL]\""
+
+all:
+	gcc open-unlink.c -o open-unlink
+	gcc create-read.c -o create-read
+
+install: all
+	install ./efivarfs.sh $(test_objs) $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start efivarfs test ....\"" >> $(KSELFTEST)
+	echo $(TEST_STR) >> $(KSELFTEST)
+	echo "echo \"End efivarfs test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
 
 run_tests: all
 	@/bin/bash ./efivarfs.sh || echo "efivarfs selftests: [FAIL]"
-- 
1.9.1

[PATCH 08/20] selftests/firmware: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/firmware/Makefile | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/testing/selftests/firmware/Makefile b/tools/testing/selftests/firmware/Makefile
index e23cce0..d5008ec 100644
--- a/tools/testing/selftests/firmware/Makefile
+++ b/tools/testing/selftests/firmware/Makefile
@@ -19,6 +19,26 @@ fw_userhelper:
                 exit 1; \
         fi
 
+install: all
+	install ./fw_filesystem.sh ./fw_userhelper.sh $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start firmware filesystem test ....\"" >> $(KSELFTEST)
+	echo "if /bin/sh ./fw_filesystem.sh ; then \\" >> $(KSELFTEST)
+	echo "\techo \"fw_filesystem: ok \"; \\" >> $(KSELFTEST)
+	echo "else \\" >> $(KSELFTEST)
+	echo "\techo \"fw_filesystem: [FAIL] \"; \\" >> $(KSELFTEST)
+	echo "fi" >> $(KSELFTEST)
+	echo "echo \"End firmware filesystem test ....\"" >> $(KSELFTEST)
+	echo "echo \"--------------------\"" >> $(KSELFTEST)
+
+	echo "echo \"Start firmware userhelper test ....\"" >> $(KSELFTEST)
+	echo "if /bin/sh ./fw_userhelper.sh ; then \\" >> $(KSELFTEST)
+	echo "\techo \"fw_userhelper: ok\"; \\" >> $(KSELFTEST)
+	echo "else \\" >> $(KSELFTEST)
+	echo "\techo \"fw_userhelper: [FAIL] \"; \\" >> $(KSELFTEST)
+	echo "fi" >> $(KSELFTEST)
+	echo "echo \"End firmware userhelper test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all fw_filesystem fw_userhelper
 
 # Nothing to clean up.
-- 
1.9.1

[PATCH 09/20] selftests/ipc: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/ipc/Makefile | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/testing/selftests/ipc/Makefile b/tools/testing/selftests/ipc/Makefile
index 74bbefd..cd1c5af 100644
--- a/tools/testing/selftests/ipc/Makefile
+++ b/tools/testing/selftests/ipc/Makefile
@@ -18,6 +18,17 @@ else
 	echo "Not an x86 target, can't build msgque selftest"
 endif
 
+install: all
+ifeq ($(ARCH),x86)
+	install ./msgque_test $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start ipc msgque test ....\"" >> $(KSELFTEST)
+	echo "./msgque_test || echo \"ipc msgque test: [FAIL]\"" >> $(KSELFTEST)
+	echo "echo \"End msgque test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+else
+	echo "Not an x86 target, can't install ipc msgque selftests"
+endif
+
 run_tests: all
 	./msgque_test
 
-- 
1.9.1

[PATCH 10/20] selftests/kcmp: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/kcmp/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tools/testing/selftests/kcmp/Makefile b/tools/testing/selftests/kcmp/Makefile
index 8aabd82..e105619 100644
--- a/tools/testing/selftests/kcmp/Makefile
+++ b/tools/testing/selftests/kcmp/Makefile
@@ -21,6 +21,18 @@ else
 	echo "Not an x86 target, can't build kcmp selftest"
 endif
 
+install: all
+ifeq ($(ARCH),x86)
+	install ./kcmp_test $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start kcmp test ....\"" >> $(KSELFTEST)
+	echo "./kcmp_test || echo \"kcmp_test: [FAIL]\"" >> $(KSELFTEST)
+	echo "rm -f kcmp-test-file" >> $(KSELFTEST)
+	echo "echo \"End kcmp test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+else
+	echo "Not an x86 target, can't install kcmp selftests"
+endif
+
 run_tests: all
 	@./kcmp_test || echo "kcmp_test: [FAIL]"
 
-- 
1.9.1

[PATCH 11/20] selftests/memfd: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/memfd/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/testing/selftests/memfd/Makefile b/tools/testing/selftests/memfd/Makefile
index b80cd10..bf03ec03 100644
--- a/tools/testing/selftests/memfd/Makefile
+++ b/tools/testing/selftests/memfd/Makefile
@@ -2,9 +2,19 @@ CFLAGS += -D_FILE_OFFSET_BITS=64
 CFLAGS += -I../../../../include/uapi/
 CFLAGS += -I../../../../include/
 
+INSTALL_PROGS = memfd_test fuse_test run_fuse_test.sh
+
 all:
 	gcc $(CFLAGS) memfd_test.c -o memfd_test
 
+install: all build_fuse
+	install $(INSTALL_PROGS) $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start memfd test ....\"" >> $(KSELFTEST)
+	echo "./memfd_test || echo \"memfd_test: [FAIL]\"" >> $(KSELFTEST)
+	echo "./run_fuse_test.sh || echo \"fuse_test: [FAIL]\"" >> $(KSELFTEST)
+	echo "echo \"End memfd test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all
 	gcc $(CFLAGS) memfd_test.c -o memfd_test
 	@./memfd_test || echo "memfd_test: [FAIL]"
-- 
1.9.1

[PATCH 12/20] selftests/memory-hotplug: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/memory-hotplug/Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tools/testing/selftests/memory-hotplug/Makefile b/tools/testing/selftests/memory-hotplug/Makefile
index d46b8d4..3c32820 100644
--- a/tools/testing/selftests/memory-hotplug/Makefile
+++ b/tools/testing/selftests/memory-hotplug/Makefile
@@ -1,5 +1,14 @@
+TEST_STR="/bin/bash ./mem-on-off-test.sh -r 2 || echo \"memory-hotplug selftests: [FAIL]\""
+
 all:
 
+install:
+	install ./on-off-test.sh $(INSTALL_KSFT_PATH)/mem-on-off-test.sh
+	echo "\necho \"Start memory hotplug test ....\"" >> $(KSELFTEST)
+	echo $(TEST_STR) >> $(KSELFTEST)
+	echo "echo \"End memory hotplug test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests:
 	@/bin/bash ./on-off-test.sh -r 2 || echo "memory-hotplug selftests: [FAIL]"
 
-- 
1.9.1

[PATCH 13/20] selftests/mount: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/mount/Makefile | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/mount/Makefile b/tools/testing/selftests/mount/Makefile
index 337d853..3bcb112 100644
--- a/tools/testing/selftests/mount/Makefile
+++ b/tools/testing/selftests/mount/Makefile
@@ -9,6 +9,13 @@ unprivileged-remount-test: unprivileged-remount-test.c
 test_unprivileged_remount: unprivileged-remount-test
 	@if [ -f /proc/self/uid_map ] ; then ./unprivileged-remount-test ; fi
 
+install: all
+	install ./unprivileged-remount-test $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start mount test ....\"" >> $(KSELFTEST)
+	echo "if [ -f /proc/self/uid_map ] ; then ./unprivileged-remount-test ; fi" >> $(KSELFTEST)
+	echo "echo \"End mount test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all test_unprivileged_remount
 
 clean:
-- 
1.9.1

[PATCH 14/20] selftests/mqueue: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/mqueue/Makefile | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/testing/selftests/mqueue/Makefile b/tools/testing/selftests/mqueue/Makefile
index 8056e2e..b0b5ef7 100644
--- a/tools/testing/selftests/mqueue/Makefile
+++ b/tools/testing/selftests/mqueue/Makefile
@@ -2,6 +2,14 @@ all:
 	gcc -O2 mq_open_tests.c -o mq_open_tests -lrt
 	gcc -O2 -o mq_perf_tests mq_perf_tests.c -lrt -lpthread -lpopt
 
+install: all
+	install ./mq_open_tests $(INSTALL_KSFT_PATH)
+	install ./mq_perf_tests $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start mqueue test ....\"" >> $(KSELFTEST)
+	echo "./mq_open_tests /test1 || echo \"mq_open_tests: [FAIL]\"" >> $(KSELFTEST)
+	echo "./mq_perf_tests || echo \"mq_perf_tests: [FAIL]\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests:
 	@./mq_open_tests /test1 || echo "mq_open_tests: [FAIL]"
 	@./mq_perf_tests || echo "mq_perf_tests: [FAIL]"
-- 
1.9.1

[PATCH 15/20] selftests/net: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/net/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 62f22cc..988e722 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -6,11 +6,21 @@ CFLAGS = -Wall -O2 -g
 CFLAGS += -I../../../../usr/include/
 
 NET_PROGS = socket psock_fanout psock_tpacket
+INSTALL_PROGS = run_netsocktests run_afpackettests test_bpf.sh $(NET_PROGS)
 
 all: $(NET_PROGS)
 %: %.c
 	$(CC) $(CFLAGS) -o $@ $^
 
+install: all
+	install $(INSTALL_PROGS) $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start net test ....\"" >> $(KSELFTEST)
+	echo "/bin/sh ./run_netsocktests || echo \"sockettests: [FAIL]\"" >> $(KSELFTEST)
+	echo "/bin/sh ./run_afpackettests || echo \"afpackettests: [FAIL]\"" >> $(KSELFTEST)
+	echo "./test_bpf.sh" >> $(KSELFTEST)
+	echo "echo \"End net test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all
 	@/bin/sh ./run_netsocktests || echo "sockettests: [FAIL]"
 	@/bin/sh ./run_afpackettests || echo "afpackettests: [FAIL]"
-- 
1.9.1

[PATCH 16/20] selftests/ptrace: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/ptrace/Makefile | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/ptrace/Makefile b/tools/testing/selftests/ptrace/Makefile
index 47ae2d3..7826045 100644
--- a/tools/testing/selftests/ptrace/Makefile
+++ b/tools/testing/selftests/ptrace/Makefile
@@ -1,7 +1,14 @@
 CFLAGS += -iquote../../../../include/uapi -Wall
-peeksiginfo: peeksiginfo.c
 
-all: peeksiginfo
+all:
+	gcc peeksiginfo.c -o peeksiginfo
+
+install: all
+	install ./peeksiginfo $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start ptrace test ....\"" >> $(KSELFTEST)
+	echo "./peeksiginfo || echo \"peeksiginfo selftests: [FAIL]\"" >> $(KSELFTEST)
+	echo "echo \"End ptrace test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
 
 clean:
 	rm -f peeksiginfo
-- 
1.9.1

[PATCH 17/20] selftests/sysctl: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/sysctl/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/testing/selftests/sysctl/Makefile b/tools/testing/selftests/sysctl/Makefile
index 0a92ada..c8d9b2b 100644
--- a/tools/testing/selftests/sysctl/Makefile
+++ b/tools/testing/selftests/sysctl/Makefile
@@ -4,6 +4,8 @@
 # No binaries, but make sure arg-less "make" doesn't trigger "run_tests".
 all:
 
+INSTALL_PROGS = common_tests run_numerictests run_stringtests
+
 # Allow specific tests to be selected.
 test_num:
 	@/bin/sh ./run_numerictests
@@ -11,6 +13,14 @@ test_num:
 test_string:
 	@/bin/sh ./run_stringtests
 
+install: all
+	install $(INSTALL_PROGS) $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start sysctl test ....\"" >> $(KSELFTEST)
+	echo "/bin/sh ./run_numerictests" >> $(KSELFTEST)
+	echo "/bin/sh ./run_stringtests" >> $(KSELFTEST)
+	echo "echo \"End sysctl test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all test_num test_string
 
 # Nothing to clean up.
-- 
1.9.1

[PATCH 18/20] selftests/timers: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/timers/Makefile | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/timers/Makefile b/tools/testing/selftests/timers/Makefile
index eb2859f..5fce7ae 100644
--- a/tools/testing/selftests/timers/Makefile
+++ b/tools/testing/selftests/timers/Makefile
@@ -1,6 +1,13 @@
 all:
 	gcc posix_timers.c -o posix_timers -lrt
 
+install: all
+	install ./posix_timers $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start timers test ....\"" >> $(KSELFTEST)
+	echo "./posix_timers" >> $(KSELFTEST)
+	echo "echo \"End timers test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all
 	./posix_timers
 
-- 
1.9.1

[PATCH 19/20] selftests/vm: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/vm/Makefile | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile
index 4c4b1f6..254ce92 100644
--- a/tools/testing/selftests/vm/Makefile
+++ b/tools/testing/selftests/vm/Makefile
@@ -9,6 +9,13 @@ all: $(BINARIES)
 %: %.c
 	$(CC) $(CFLAGS) -o $@ $^
 
+install: all
+	install run_vmtests $(BINARIES) $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start vm test ....\"" >> $(KSELFTEST)
+	echo "/bin/sh ./run_vmtests || echo \"vmtests: [FAIL]\"" >> $(KSELFTEST)
+	echo "echo \"End vm test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all
 	@/bin/sh ./run_vmtests || (echo "vmtests: [FAIL]"; exit 1)
 
-- 
1.9.1

[PATCH 20/20] selftests/user: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/user/Makefile | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/testing/selftests/user/Makefile b/tools/testing/selftests/user/Makefile
index 12c9d15..c8e3863 100644
--- a/tools/testing/selftests/user/Makefile
+++ b/tools/testing/selftests/user/Makefile
@@ -3,5 +3,12 @@
 # No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
 all:
 
+install: all
+	install ./test_user_copy.sh $(INSTALL_KSFT_PATH)
+	echo "\necho \"Start user copy test ....\"" >> $(KSELFTEST)
+	echo "./test_user_copy.sh" >> $(KSELFTEST)
+	echo "echo \"End user copy test ....\"" >> $(KSELFTEST)
+	echo "echo \"==============================\"" >> $(KSELFTEST)
+
 run_tests: all
 	./test_user_copy.sh
-- 
1.9.1

Re: [PATCH 00/20] kselftest install target feature

[Kees Cook]

On Tue, Nov 4, 2014 at 9:10 AM, Shuah Khan <shuahkh@osg.samsung.com> wrote:
> This patch series adds a new kselftest_install make target
> to enable selftest install. When make kselftest_install is
> run, selftests are installed on the system. A new install
> target is added to selftests Makefile which will install
> targets for the tests that are specified in INSTALL_TARGETS.
> During install, a script is generated to run tests that are
> installed. This script will be installed in the selftest install
> directory. Individual test Makefiles are changed to add to the
> script. This will allow new tests to add install and run test
> commands to the generated kselftest script.

I'm all for making the self tests more available, but I don't think
this is the right approach. My primary objection is that it creates a
second way to run tests, and that means any changes and additions need
to be updated in two places. I'd much rather just maintain the single
"make" targets instead. Having "make" available on the target device
doesn't seem too bad to me. Is there a reason that doesn't work for
your situation?

I would, however, like to see some better standardization of the test
"framework" that we've got in there already. (For example, some
failures fail the "make", some don't, there are various reporting
methods for success/failure depending on the test, etc.)

-Kees

> Approach:
>
> make kselftest_target:
> -- exports kselftest INSTALL_KSFT_PATH
>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> -- exports path for ksefltest.sh
> -- runs selftests make install target:
>
> selftests make install target
> -- creates kselftest.sh script in install install dir
> -- runs install targets for all INSTALL_TARGETS
>
> Individual test make install targets:
> -- install test programs and/or scripts in install dir
> -- append to the ksefltest.sh file to add commands to run test
>
> Shuah Khan (20):
>   selftests/user: move test out of Makefile into a shell script
>   selftests/net: move test out of Makefile into a shell script
>   kbuild: kselftest_install - add a new make target to install selftests
>   selftests: add install target to enable installing selftests
>   selftests/breakpoints: add install target to enable installing test
>   selftests/cpu-hotplug: add install target to enable installing test
>   selftests/efivarfs: add install target to enable installing test
>   selftests/firmware: add install target to enable installing test
>   selftests/ipc: add install target to enable installing test
>   selftests/kcmp: add install target to enable installing test
>   selftests/memfd: add install target to enable installing test
>   selftests/memory-hotplug: add install target to enable installing test
>   selftests/mount: add install target to enable installing test
>   selftests/mqueue: add install target to enable installing test
>   selftests/net: add install target to enable installing test
>   selftests/ptrace: add install target to enable installing test
>   selftests/sysctl: add install target to enable installing test
>   selftests/timers: add install target to enable installing test
>   selftests/vm: add install target to enable installing test
>   selftests/user: add install target to enable installing test
>
>  Makefile                                        | 17 +++++++++++++++++
>  tools/testing/selftests/Makefile                | 14 ++++++++++++++
>  tools/testing/selftests/breakpoints/Makefile    | 12 ++++++++++++
>  tools/testing/selftests/cpu-hotplug/Makefile    |  9 +++++++++
>  tools/testing/selftests/efivarfs/Makefile       | 13 ++++++++++++-
>  tools/testing/selftests/firmware/Makefile       | 20 ++++++++++++++++++++
>  tools/testing/selftests/ipc/Makefile            | 11 +++++++++++
>  tools/testing/selftests/kcmp/Makefile           | 12 ++++++++++++
>  tools/testing/selftests/memfd/Makefile          | 10 ++++++++++
>  tools/testing/selftests/memory-hotplug/Makefile |  9 +++++++++
>  tools/testing/selftests/mount/Makefile          |  7 +++++++
>  tools/testing/selftests/mqueue/Makefile         |  8 ++++++++
>  tools/testing/selftests/net/Makefile            | 18 +++++++++++-------
>  tools/testing/selftests/net/test_bpf.sh         | 10 ++++++++++
>  tools/testing/selftests/ptrace/Makefile         | 11 +++++++++--
>  tools/testing/selftests/sysctl/Makefile         | 10 ++++++++++
>  tools/testing/selftests/timers/Makefile         |  7 +++++++
>  tools/testing/selftests/user/Makefile           | 15 ++++++++-------
>  tools/testing/selftests/user/test_user_copy.sh  | 10 ++++++++++
>  tools/testing/selftests/vm/Makefile             |  7 +++++++
>  20 files changed, 213 insertions(+), 17 deletions(-)
>  create mode 100755 tools/testing/selftests/net/test_bpf.sh
>  create mode 100755 tools/testing/selftests/user/test_user_copy.sh
>
> --
> 1.9.1
>



-- 
Kees Cook
Chrome OS Security

Re: [PATCH 02/20] selftests/net: move test out of Makefile into a shell script

[David Miller]

From: Shuah Khan <shuahkh@osg.samsung.com>
Date: Tue,  4 Nov 2014 10:10:58 -0700

> Currently bpf test run from the Makefile. Move it out of the
> Makefile to be run from a shell script to allow the test to
> be run as stand-alone test, in addition to allowing the test
> run from a make target.
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>

Acked-by: David S. Miller <davem@davemloft.net>

Re: [PATCH 00/20] kselftest install target feature

[Shuah Khan]

On 11/04/2014 12:22 PM, Kees Cook wrote:
> On Tue, Nov 4, 2014 at 9:10 AM, Shuah Khan <shuahkh@osg.samsung.com> wrote:
>> This patch series adds a new kselftest_install make target
>> to enable selftest install. When make kselftest_install is
>> run, selftests are installed on the system. A new install
>> target is added to selftests Makefile which will install
>> targets for the tests that are specified in INSTALL_TARGETS.
>> During install, a script is generated to run tests that are
>> installed. This script will be installed in the selftest install
>> directory. Individual test Makefiles are changed to add to the
>> script. This will allow new tests to add install and run test
>> commands to the generated kselftest script.
> 
> I'm all for making the self tests more available, but I don't think
> this is the right approach. My primary objection is that it creates a
> second way to run tests, and that means any changes and additions need
> to be updated in two places. I'd much rather just maintain the single
> "make" targets instead. Having "make" available on the target device
> doesn't seem too bad to me. Is there a reason that doesn't work for
> your situation?

Kees,

My primary objective is to provide a way to install selftests for a
specific kernel release. This will allow developers to run tests for
a specific release and look for regressions. Adding an install target
will also help support local execution of tests in a virtualized
environments. In some cases such as qemu, it is not practical to
expect the target to have support for "make". Once tests are installed
to be run outside the git environment, we need a master script that
can run the tests. Hence the need for a master script that can run
tests.

We have the ability to run all tests via make kselftest target or
run a specific test using the individual test's run_tests target.
Both of above are necessary to support running tests from the tree.
Embedding run_tests logic in the makefiles doesn't work very well
in the long run.

We also need a way to run them outside tree. I agree with you that
the way I added the script generation, duplicates the code in individual
run_tests targets and that changes/updates need to be made in both
places.

Would you be ok with the approach if I fixed the duplicating
problem? I can address the duplication concern easily.

> 
> I would, however, like to see some better standardization of the test
> "framework" that we've got in there already. (For example, some
> failures fail the "make", some don't, there are various reporting
> methods for success/failure depending on the test, etc.)

This is being addressed and I have the framework in linux-kselftest
git next branch at the moment. I do think the above work is part of
addressing the larger framework issues such as being able to run tests
on a target system that might not have "make" support and makes it
easier to use.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH 00/20] kselftest install target feature

[Kees Cook]

On Tue, Nov 4, 2014 at 3:45 PM, Shuah Khan <shuahkh@osg.samsung.com> wrote:
> On 11/04/2014 12:22 PM, Kees Cook wrote:
>> On Tue, Nov 4, 2014 at 9:10 AM, Shuah Khan <shuahkh@osg.samsung.com> wrote:
>>> This patch series adds a new kselftest_install make target
>>> to enable selftest install. When make kselftest_install is
>>> run, selftests are installed on the system. A new install
>>> target is added to selftests Makefile which will install
>>> targets for the tests that are specified in INSTALL_TARGETS.
>>> During install, a script is generated to run tests that are
>>> installed. This script will be installed in the selftest install
>>> directory. Individual test Makefiles are changed to add to the
>>> script. This will allow new tests to add install and run test
>>> commands to the generated kselftest script.
>>
>> I'm all for making the self tests more available, but I don't think
>> this is the right approach. My primary objection is that it creates a
>> second way to run tests, and that means any changes and additions need
>> to be updated in two places. I'd much rather just maintain the single
>> "make" targets instead. Having "make" available on the target device
>> doesn't seem too bad to me. Is there a reason that doesn't work for
>> your situation?
>
> Kees,
>
> My primary objective is to provide a way to install selftests for a
> specific kernel release. This will allow developers to run tests for
> a specific release and look for regressions. Adding an install target
> will also help support local execution of tests in a virtualized
> environments. In some cases such as qemu, it is not practical to
> expect the target to have support for "make". Once tests are installed
> to be run outside the git environment, we need a master script that
> can run the tests. Hence the need for a master script that can run
> tests.
>
> We have the ability to run all tests via make kselftest target or
> run a specific test using the individual test's run_tests target.
> Both of above are necessary to support running tests from the tree.
> Embedding run_tests logic in the makefiles doesn't work very well
> in the long run.
>
> We also need a way to run them outside tree. I agree with you that
> the way I added the script generation, duplicates the code in individual
> run_tests targets and that changes/updates need to be made in both
> places.
>
> Would you be ok with the approach if I fixed the duplicating
> problem? I can address the duplication concern easily.

Yeah, getting rid of duplication would be much preferred. Thanks!

-Kees

>
>>
>> I would, however, like to see some better standardization of the test
>> "framework" that we've got in there already. (For example, some
>> failures fail the "make", some don't, there are various reporting
>> methods for success/failure depending on the test, etc.)
>
> This is being addressed and I have the framework in linux-kselftest
> git next branch at the moment. I do think the above work is part of
> addressing the larger framework issues such as being able to run tests
> on a target system that might not have "make" support and makes it
> easier to use.
>
> thanks,
> -- Shuah
>
>
> --
> Shuah Khan
> Sr. Linux Kernel Developer
> Samsung Research America (Silicon Valley)
> shuahkh@osg.samsung.com | (970) 217-8978



-- 
Kees Cook
Chrome OS Security