[PATCH v2 00/19] kselftest install target feature

[PATCH v2 00/19] kselftest install target feature

[Shuah Khan]

This patch series adds a new kselftest_install make target
to enable selftest install. When make kselftest_install is
run, selftests are installed on the system. A new install
target is added to selftests Makefile which will install
targets for the tests that are specified in INSTALL_TARGETS.
During install, a script is generated to run tests that are
installed. This script will be installed in the selftest install
directory. Individual test Makefiles are changed to add to the
script. This will allow new tests to add install and run test
commands to the generated kselftest script. kselftest target
now depends on kselftest_install and runs the generated kselftest
script to reduce duplicate work and for common look and feel when
running tests.

This v2 series addresses the duplicate code in install and
run_tests targets in individual test Makefiles.
Reference: https://lkml.org/lkml/2014/11/4/707

Dropped the following patch from the series since it has been
Acked and I queued it up for 3.19-rc1.

selftests/net: move test out of Makefile into a shell script
Reference: https://lkml.org/lkml/2014/11/4/794

Approach:

make kselftest_target:
-- exports kselftest INSTALL_KSFT_PATH
   default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
-- exports path for ksefltest.sh
-- runs selftests make install target:

selftests make install target
-- creates kselftest.sh script in install install dir
-- runs install targets for all INSTALL_TARGETS
   (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
          to not add more content to patch v1 series. This work
          will happen soon. In this series these two targets are
          run after running the generated kselftest script, without
          any regression in the way these tests are run with
          "make kselftest" prior to this work.)
-- install target can be run only from top level source dir.

Individual test make install targets:
-- install test programs and/or scripts in install dir
-- append to the ksefltest.sh file to add commands to run test
-- install target can be run only from top level source dir.

Adds the following new ways to initiate selftests:
-- Installing and running kselftest from install directory
   by running  "make kselftest"
-- Running kselftest script from install directory

Maintains the following ways to run tests:
-- make -C tools/testing/selftests run_tests
-- make -C tools/testing/selftests TARGETS=target run_tests
   Ability specify targets: e.g TARGETS=net
-- make run_tests from tools/testing/selftests
-- make run_tests from individual test directories:
   e.g: make run_tests in tools/testing/selftests/breakpoints

Shuah Khan (19):
  selftests/user: move test out of Makefile into a shell script
  kbuild: kselftest_install - add a new make target to install selftests
  selftests: add install target to enable installing selftests
  selftests/breakpoints: add install target to enable installing test
  selftests/cpu-hotplug: add install target to enable installing test
  selftests/efivarfs: add install target to enable installing test
  selftests/firmware: add install target to enable installing test
  selftests/ipc: add install target to enable installing test
  selftests/kcmp: add install target to enable installing test
  selftests/memfd: add install target to enable installing test
  selftests/memory-hotplug: add install target to enable installing test
  selftests/mount: add install target to enable installing test
  selftests/mqueue: add install target to enable installing test
  selftests/net: add install target to enable installing test
  selftests/ptrace: add install target to enable installing test
  selftests/sysctl: add install target to enable installing test
  selftests/timers: add install target to enable installing test
  selftests/vm: add install target to enable installing test
  selftests/user: add install target to enable installing test

 Makefile                                           |  21 +-
 tools/testing/selftests/Makefile                   |  31 ++-
 tools/testing/selftests/breakpoints/Makefile       |  22 +-
 tools/testing/selftests/cpu-hotplug/Makefile       |  17 +-
 .../selftests/cpu-hotplug/cpu-on-off-test.sh       | 269 +++++++++++++++++++++
 tools/testing/selftests/cpu-hotplug/on-off-test.sh | 269 ---------------------
 tools/testing/selftests/efivarfs/Makefile          |  19 +-
 tools/testing/selftests/firmware/Makefile          |  49 ++--
 tools/testing/selftests/ipc/Makefile               |  22 +-
 tools/testing/selftests/kcmp/Makefile              |  22 +-
 tools/testing/selftests/memfd/Makefile             |  20 +-
 tools/testing/selftests/memory-hotplug/Makefile    |  17 +-
 .../selftests/memory-hotplug/mem-on-off-test.sh    | 238 ++++++++++++++++++
 .../selftests/memory-hotplug/on-off-test.sh        | 238 ------------------
 tools/testing/selftests/mount/Makefile             |  15 +-
 tools/testing/selftests/mqueue/Makefile            |  20 +-
 tools/testing/selftests/net/Makefile               |  23 +-
 tools/testing/selftests/ptrace/Makefile            |  19 +-
 tools/testing/selftests/sysctl/Makefile            |  23 +-
 tools/testing/selftests/timers/Makefile            |  15 +-
 tools/testing/selftests/user/Makefile              |  21 +-
 tools/testing/selftests/user/test_user_copy.sh     |  10 +
 tools/testing/selftests/vm/Makefile                |  14 +-
 23 files changed, 853 insertions(+), 561 deletions(-)
 create mode 100644 tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
 delete mode 100644 tools/testing/selftests/cpu-hotplug/on-off-test.sh
 create mode 100644 tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
 delete mode 100644 tools/testing/selftests/memory-hotplug/on-off-test.sh
 create mode 100755 tools/testing/selftests/user/test_user_copy.sh

-- 
1.9.1

[PATCH v2 01/19] selftests/user: move test out of Makefile into a shell script

[Shuah Khan]

Currently user copy test is run from the Makefile. Move it out
of the Makefile to be run from a shell script to allow the test
to be run as stand-alone test, in addition to allowing the test
run from a make target.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/user/Makefile          |  8 +-------
 tools/testing/selftests/user/test_user_copy.sh | 10 ++++++++++
 2 files changed, 11 insertions(+), 7 deletions(-)
 create mode 100755 tools/testing/selftests/user/test_user_copy.sh

diff --git a/tools/testing/selftests/user/Makefile b/tools/testing/selftests/user/Makefile
index 396255b..12c9d15 100644
--- a/tools/testing/selftests/user/Makefile
+++ b/tools/testing/selftests/user/Makefile
@@ -4,10 +4,4 @@
 all:
 
 run_tests: all
-	@if /sbin/modprobe test_user_copy ; then \
-		rmmod test_user_copy; \
-		echo "user_copy: ok"; \
-	else \
-		echo "user_copy: [FAIL]"; \
-		exit 1; \
-	fi
+	./test_user_copy.sh
diff --git a/tools/testing/selftests/user/test_user_copy.sh b/tools/testing/selftests/user/test_user_copy.sh
new file mode 100755
index 0000000..350107f
--- /dev/null
+++ b/tools/testing/selftests/user/test_user_copy.sh
@@ -0,0 +1,10 @@
+#!/bin/sh
+# Runs copy_to/from_user infrastructure using test_user_copy kernel module
+
+if /sbin/modprobe -q test_user_copy; then
+	/sbin/modprobe -q -r test_user_copy
+	echo "user_copy: ok"
+else
+	echo "user_copy: [FAIL]"
+	exit 1
+fi
-- 
1.9.1

[PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

Add a new make target to install to install kernel selftests.
This new target will build and install selftests. kselftest
target now depends on kselftest_install and runs the generated
kselftest script to reduce duplicate work and for common look
and feel when running tests.

Approach:

make kselftest_target:
-- exports kselftest INSTALL_KSFT_PATH
   default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
-- exports path for ksefltest.sh
-- runs selftests make install target:

selftests make install target
-- creates kselftest.sh script in install install dir
-- runs install targets for all INSTALL_TARGETS
   (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
          to not add more content to patch v1 series. This work
          will happen soon. In this series these two targets are
          run after running the generated kselftest script, without
          any regression in the way these tests are run with
          "make kselftest" prior to this work.)
-- install target can be run only from top level source dir.

Individual test make install targets:
-- install test programs and/or scripts in install dir
-- append to the ksefltest.sh file to add commands to run test
-- install target can be run only from top level source dir.

Adds the following new ways to initiate selftests:
-- Installing and running kselftest from install directory
   by running  "make kselftest"
-- Running kselftest script from install directory

Maintains the following ways to run tests:
-- make -C tools/testing/selftests run_tests
-- make -C tools/testing/selftests TARGETS=target run_tests
   Ability specify targets: e.g TARGETS=net
-- make run_tests from tools/testing/selftests
-- make run_tests from individual test directories:
   e.g: make run_tests in tools/testing/selftests/breakpoints

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 Makefile | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 05d67af..ccbd2e1 100644
--- a/Makefile
+++ b/Makefile
@@ -1071,12 +1071,26 @@ headers_check: headers_install
 	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
 
 # ---------------------------------------------------------------------------
-# Kernel selftest
+# Kernel selftest targets
+
+PHONY += __kselftest_configure
+INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
+export INSTALL_KSFT_PATH
+KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
+export KSELFTEST
 
 PHONY += kselftest
-kselftest:
+kselftest: kselftest_install
 	$(Q)$(MAKE) -C tools/testing/selftests run_tests
 
+# Kernel selftest install
+
+PHONY += kselftest_install
+kselftest_install: __kselftest_configure
+	@rm -rf $(INSTALL_KSFT_PATH)
+	@mkdir -p $(INSTALL_KSFT_PATH)
+	$(Q)$(MAKE) -C tools/testing/selftests install
+
 # ---------------------------------------------------------------------------
 # Modules
 
@@ -1285,6 +1299,9 @@ help:
 	@echo  '                    Build, install, and boot kernel before'
 	@echo  '                    running kselftest on it'
 	@echo  ''
+	@echo  '  kselftest_install - Install Kselftests to INSTALL_KSFT_PATH'
+	@echo  '                      default: $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)'
+	@echo  ''
 	@echo  'Kernel packaging:'
 	@$(MAKE) $(build)=$(package-dir) help
 	@echo  ''
-- 
1.9.1

[PATCH v2 03/19] selftests: add install target to enable installing selftests

[Shuah Khan]

Add a new make target to enable installing selftests. This
new target will call install targets for the tests that are
specified in INSTALL_TARGETS. During install, a script is
generated to run tests that are installed. This script will
be installed in the selftest install directory. Individual
test Makefiles are changed to add to the script. This will
allow new tests to add install and run test commands to the
generated kselftest script. run_tests target runs the
generated kselftest script to run tests when it is initiated
from from "make kselftest" from top level source directory.

Approach:

make kselftest_target:
-- exports kselftest INSTALL_KSFT_PATH
   default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
-- exports path for ksefltest.sh
-- runs selftests make install target:

selftests make install target
-- creates kselftest.sh script in install install dir
-- runs install targets for all INSTALL_TARGETS
   (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
          to not add more content to patch v1 series. This work
          will happen soon. In this series these two targets are
          run after running the generated kselftest script, without
          any regression in the way these tests are run with
          "make kselftest" prior to this work.)
-- install target can be run only from top level source dir.

Individual test make install targets:
-- install test programs and/or scripts in install dir
-- append to the ksefltest.sh file to add commands to run test
-- install target can be run only from top level source dir.

Adds the following new ways to initiate selftests:
-- Installing and running kselftest from install directory
   by running  "make kselftest"
-- Running kselftest script from install directory

Maintains the following ways to run tests:
-- make -C tools/testing/selftests run_tests
-- make -C tools/testing/selftests TARGETS=target run_tests
   Ability specify targets: e.g TARGETS=net
-- make run_tests from tools/testing/selftests
-- make run_tests from individual test directories:
   e.g: make run_tests in tools/testing/selftests/breakpoints

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/Makefile | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 45f145c..b9bdc1d 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -16,6 +16,10 @@ TARGETS += sysctl
 TARGETS += firmware
 TARGETS += ftrace
 
+INSTALL_TARGETS = breakpoints cpu-hotplug efivarfs firmware ipc
+INSTALL_TARGETS += kcmp memfd memory-hotplug mqueue mount net
+INSTALL_TARGETS += ptrace sysctl timers user vm
+
 TARGETS_HOTPLUG = cpu-hotplug
 TARGETS_HOTPLUG += memory-hotplug
 
@@ -24,10 +28,35 @@ all:
 		make -C $$TARGET; \
 	done;
 
-run_tests: all
+install:
+ifdef INSTALL_KSFT_PATH
+	make all
+	@echo #!/bin/sh\n# Kselftest Run Tests .... >> $(KSELFTEST)
+	@echo # This file is generated during kselftest_install >> $(KSELFTEST)
+	@echo # Please don't change it !!\n  >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+	for TARGET in $(INSTALL_TARGETS); do \
+		echo Installing $$TARGET; \
+		make -C $$TARGET install; \
+	done;
+	chmod +x $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
+run_tests:
+ifdef INSTALL_KSFT_PATH
+	@cd $(INSTALL_KSFT_PATH); ./kselftest.sh; cd -
+# TODO: include ftrace and powerpc in install targets
+	for TARGET in ftrace powerpc; do \
+		make -C $$TARGET run_tests; \
+	done;
+else
+	make all
 	for TARGET in $(TARGETS); do \
 		make -C $$TARGET run_tests; \
 	done;
+endif
 
 hotplug:
 	for TARGET in $(TARGETS_HOTPLUG); do \
-- 
1.9.1

[PATCH v2 04/19] selftests/breakpoints: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/breakpoints/Makefile | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/breakpoints/Makefile b/tools/testing/selftests/breakpoints/Makefile
index e18b42b..9a70aeb 100644
--- a/tools/testing/selftests/breakpoints/Makefile
+++ b/tools/testing/selftests/breakpoints/Makefile
@@ -8,6 +8,7 @@ ifeq ($(ARCH),x86_64)
 	ARCH := x86
 endif
 
+TEST_STR = ./breakpoint_test || echo breakpoints selftests: [FAIL]
 
 all:
 ifeq ($(ARCH),x86)
@@ -16,8 +17,25 @@ else
 	echo "Not an x86 target, can't build breakpoints selftests"
 endif
 
-run_tests:
-	@./breakpoint_test || echo "breakpoints selftests: [FAIL]"
+install:
+ifdef INSTALL_KSFT_PATH
+ifeq ($(ARCH),x86)
+	install ./breakpoint_test $(INSTALL_KSFT_PATH)
+	@echo echo Start breakpoints test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End breakpoints test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Not an x86 target, unable to install breakpoints selftests
+endif
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
+run_tests: all
+ifeq ($(ARCH),x86)
+	@$(TEST_STR)
+endif
 
 clean:
 	rm -fr breakpoint_test
-- 
1.9.1

[PATCH v2 05/19] selftests/cpu-hotplug: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/cpu-hotplug/Makefile       |  17 +-
 .../selftests/cpu-hotplug/cpu-on-off-test.sh       | 269 +++++++++++++++++++++
 tools/testing/selftests/cpu-hotplug/on-off-test.sh | 269 ---------------------
 3 files changed, 284 insertions(+), 271 deletions(-)
 create mode 100644 tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
 delete mode 100644 tools/testing/selftests/cpu-hotplug/on-off-test.sh

diff --git a/tools/testing/selftests/cpu-hotplug/Makefile b/tools/testing/selftests/cpu-hotplug/Makefile
index e9c28d8..1bdcfc3 100644
--- a/tools/testing/selftests/cpu-hotplug/Makefile
+++ b/tools/testing/selftests/cpu-hotplug/Makefile
@@ -1,9 +1,22 @@
+TEST_STR=/bin/bash ./cpu-on-off-test.sh || echo cpu-hotplug selftests: [FAIL]
+
 all:
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./cpu-on-off-test.sh $(INSTALL_KSFT_PATH)/cpu-on-off-test.sh
+	@echo echo Start cpu hotplug test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End cpu hotplug test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests:
-	@/bin/bash ./on-off-test.sh || echo "cpu-hotplug selftests: [FAIL]"
+	@$(TEST_STR)
 
 run_full_test:
-	@/bin/bash ./on-off-test.sh -a || echo "cpu-hotplug selftests: [FAIL]"
+	@/bin/bash ./cpu-on-off-test.sh -a || echo "cpu-hotplug selftests: [FAIL]"
 
 clean:
diff --git a/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh b/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
new file mode 100644
index 0000000..98b1d65
--- /dev/null
+++ b/tools/testing/selftests/cpu-hotplug/cpu-on-off-test.sh
@@ -0,0 +1,269 @@
+#!/bin/bash
+
+SYSFS=
+
+prerequisite()
+{
+	msg="skip all tests:"
+
+	if [ $UID != 0 ]; then
+		echo $msg must be run as root >&2
+		exit 0
+	fi
+
+	taskset -p 01 $$
+
+	SYSFS=`mount -t sysfs | head -1 | awk '{ print $3 }'`
+
+	if [ ! -d "$SYSFS" ]; then
+		echo $msg sysfs is not mounted >&2
+		exit 0
+	fi
+
+	if ! ls $SYSFS/devices/system/cpu/cpu* > /dev/null 2>&1; then
+		echo $msg cpu hotplug is not supported >&2
+		exit 0
+	fi
+
+	echo "CPU online/offline summary:"
+	online_cpus=`cat $SYSFS/devices/system/cpu/online`
+	online_max=${online_cpus##*-}
+	echo -e "\t Cpus in online state: $online_cpus"
+
+	offline_cpus=`cat $SYSFS/devices/system/cpu/offline`
+	if [[ "a$offline_cpus" = "a" ]]; then
+		offline_cpus=0
+	else
+		offline_max=${offline_cpus##*-}
+	fi
+	echo -e "\t Cpus in offline state: $offline_cpus"
+}
+
+#
+# list all hot-pluggable CPUs
+#
+hotpluggable_cpus()
+{
+	local state=${1:-.\*}
+
+	for cpu in $SYSFS/devices/system/cpu/cpu*; do
+		if [ -f $cpu/online ] && grep -q $state $cpu/online; then
+			echo ${cpu##/*/cpu}
+		fi
+	done
+}
+
+hotplaggable_offline_cpus()
+{
+	hotpluggable_cpus 0
+}
+
+hotpluggable_online_cpus()
+{
+	hotpluggable_cpus 1
+}
+
+cpu_is_online()
+{
+	grep -q 1 $SYSFS/devices/system/cpu/cpu$1/online
+}
+
+cpu_is_offline()
+{
+	grep -q 0 $SYSFS/devices/system/cpu/cpu$1/online
+}
+
+online_cpu()
+{
+	echo 1 > $SYSFS/devices/system/cpu/cpu$1/online
+}
+
+offline_cpu()
+{
+	echo 0 > $SYSFS/devices/system/cpu/cpu$1/online
+}
+
+online_cpu_expect_success()
+{
+	local cpu=$1
+
+	if ! online_cpu $cpu; then
+		echo $FUNCNAME $cpu: unexpected fail >&2
+	elif ! cpu_is_online $cpu; then
+		echo $FUNCNAME $cpu: unexpected offline >&2
+	fi
+}
+
+online_cpu_expect_fail()
+{
+	local cpu=$1
+
+	if online_cpu $cpu 2> /dev/null; then
+		echo $FUNCNAME $cpu: unexpected success >&2
+	elif ! cpu_is_offline $cpu; then
+		echo $FUNCNAME $cpu: unexpected online >&2
+	fi
+}
+
+offline_cpu_expect_success()
+{
+	local cpu=$1
+
+	if ! offline_cpu $cpu; then
+		echo $FUNCNAME $cpu: unexpected fail >&2
+	elif ! cpu_is_offline $cpu; then
+		echo $FUNCNAME $cpu: unexpected offline >&2
+	fi
+}
+
+offline_cpu_expect_fail()
+{
+	local cpu=$1
+
+	if offline_cpu $cpu 2> /dev/null; then
+		echo $FUNCNAME $cpu: unexpected success >&2
+	elif ! cpu_is_online $cpu; then
+		echo $FUNCNAME $cpu: unexpected offline >&2
+	fi
+}
+
+error=-12
+allcpus=0
+priority=0
+online_cpus=0
+online_max=0
+offline_cpus=0
+offline_max=0
+
+while getopts e:ahp: opt; do
+	case $opt in
+	e)
+		error=$OPTARG
+		;;
+	a)
+		allcpus=1
+		;;
+	h)
+		echo "Usage $0 [ -a ] [ -e errno ] [ -p notifier-priority ]"
+		echo -e "\t default offline one cpu"
+		echo -e "\t run with -a option to offline all cpus"
+		exit
+		;;
+	p)
+		priority=$OPTARG
+		;;
+	esac
+done
+
+if ! [ "$error" -ge -4095 -a "$error" -lt 0 ]; then
+	echo "error code must be -4095 <= errno < 0" >&2
+	exit 1
+fi
+
+prerequisite
+
+#
+# Safe test (default) - offline and online one cpu
+#
+if [ $allcpus -eq 0 ]; then
+	echo "Limited scope test: one hotplug cpu"
+	echo -e "\t (leaves cpu in the original state):"
+	echo -e "\t online to offline to online: cpu $online_max"
+	offline_cpu_expect_success $online_max
+	online_cpu_expect_success $online_max
+
+	if [[ $offline_cpus -gt 0 ]]; then
+		echo -e "\t offline to online to offline: cpu $offline_max"
+		online_cpu_expect_success $offline_max
+		offline_cpu_expect_success $offline_max
+	fi
+	exit 0
+else
+	echo "Full scope test: all hotplug cpus"
+	echo -e "\t online all offline cpus"
+	echo -e "\t offline all online cpus"
+	echo -e "\t online all offline cpus"
+fi
+
+#
+# Online all hot-pluggable CPUs
+#
+for cpu in `hotplaggable_offline_cpus`; do
+	online_cpu_expect_success $cpu
+done
+
+#
+# Offline all hot-pluggable CPUs
+#
+for cpu in `hotpluggable_online_cpus`; do
+	offline_cpu_expect_success $cpu
+done
+
+#
+# Online all hot-pluggable CPUs again
+#
+for cpu in `hotplaggable_offline_cpus`; do
+	online_cpu_expect_success $cpu
+done
+
+#
+# Test with cpu notifier error injection
+#
+
+DEBUGFS=`mount -t debugfs | head -1 | awk '{ print $3 }'`
+NOTIFIER_ERR_INJECT_DIR=$DEBUGFS/notifier-error-inject/cpu
+
+prerequisite_extra()
+{
+	msg="skip extra tests:"
+
+	/sbin/modprobe -q -r cpu-notifier-error-inject
+	/sbin/modprobe -q cpu-notifier-error-inject priority=$priority
+
+	if [ ! -d "$DEBUGFS" ]; then
+		echo $msg debugfs is not mounted >&2
+		exit 0
+	fi
+
+	if [ ! -d $NOTIFIER_ERR_INJECT_DIR ]; then
+		echo $msg cpu-notifier-error-inject module is not available >&2
+		exit 0
+	fi
+}
+
+prerequisite_extra
+
+#
+# Offline all hot-pluggable CPUs
+#
+echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_DOWN_PREPARE/error
+for cpu in `hotpluggable_online_cpus`; do
+	offline_cpu_expect_success $cpu
+done
+
+#
+# Test CPU hot-add error handling (offline => online)
+#
+echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_UP_PREPARE/error
+for cpu in `hotplaggable_offline_cpus`; do
+	online_cpu_expect_fail $cpu
+done
+
+#
+# Online all hot-pluggable CPUs
+#
+echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_UP_PREPARE/error
+for cpu in `hotplaggable_offline_cpus`; do
+	online_cpu_expect_success $cpu
+done
+
+#
+# Test CPU hot-remove error handling (online => offline)
+#
+echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_DOWN_PREPARE/error
+for cpu in `hotpluggable_online_cpus`; do
+	offline_cpu_expect_fail $cpu
+done
+
+echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_DOWN_PREPARE/error
+/sbin/modprobe -q -r cpu-notifier-error-inject
diff --git a/tools/testing/selftests/cpu-hotplug/on-off-test.sh b/tools/testing/selftests/cpu-hotplug/on-off-test.sh
deleted file mode 100644
index 98b1d65..0000000
--- a/tools/testing/selftests/cpu-hotplug/on-off-test.sh
+++ /dev/null
@@ -1,269 +0,0 @@
-#!/bin/bash
-
-SYSFS=
-
-prerequisite()
-{
-	msg="skip all tests:"
-
-	if [ $UID != 0 ]; then
-		echo $msg must be run as root >&2
-		exit 0
-	fi
-
-	taskset -p 01 $$
-
-	SYSFS=`mount -t sysfs | head -1 | awk '{ print $3 }'`
-
-	if [ ! -d "$SYSFS" ]; then
-		echo $msg sysfs is not mounted >&2
-		exit 0
-	fi
-
-	if ! ls $SYSFS/devices/system/cpu/cpu* > /dev/null 2>&1; then
-		echo $msg cpu hotplug is not supported >&2
-		exit 0
-	fi
-
-	echo "CPU online/offline summary:"
-	online_cpus=`cat $SYSFS/devices/system/cpu/online`
-	online_max=${online_cpus##*-}
-	echo -e "\t Cpus in online state: $online_cpus"
-
-	offline_cpus=`cat $SYSFS/devices/system/cpu/offline`
-	if [[ "a$offline_cpus" = "a" ]]; then
-		offline_cpus=0
-	else
-		offline_max=${offline_cpus##*-}
-	fi
-	echo -e "\t Cpus in offline state: $offline_cpus"
-}
-
-#
-# list all hot-pluggable CPUs
-#
-hotpluggable_cpus()
-{
-	local state=${1:-.\*}
-
-	for cpu in $SYSFS/devices/system/cpu/cpu*; do
-		if [ -f $cpu/online ] && grep -q $state $cpu/online; then
-			echo ${cpu##/*/cpu}
-		fi
-	done
-}
-
-hotplaggable_offline_cpus()
-{
-	hotpluggable_cpus 0
-}
-
-hotpluggable_online_cpus()
-{
-	hotpluggable_cpus 1
-}
-
-cpu_is_online()
-{
-	grep -q 1 $SYSFS/devices/system/cpu/cpu$1/online
-}
-
-cpu_is_offline()
-{
-	grep -q 0 $SYSFS/devices/system/cpu/cpu$1/online
-}
-
-online_cpu()
-{
-	echo 1 > $SYSFS/devices/system/cpu/cpu$1/online
-}
-
-offline_cpu()
-{
-	echo 0 > $SYSFS/devices/system/cpu/cpu$1/online
-}
-
-online_cpu_expect_success()
-{
-	local cpu=$1
-
-	if ! online_cpu $cpu; then
-		echo $FUNCNAME $cpu: unexpected fail >&2
-	elif ! cpu_is_online $cpu; then
-		echo $FUNCNAME $cpu: unexpected offline >&2
-	fi
-}
-
-online_cpu_expect_fail()
-{
-	local cpu=$1
-
-	if online_cpu $cpu 2> /dev/null; then
-		echo $FUNCNAME $cpu: unexpected success >&2
-	elif ! cpu_is_offline $cpu; then
-		echo $FUNCNAME $cpu: unexpected online >&2
-	fi
-}
-
-offline_cpu_expect_success()
-{
-	local cpu=$1
-
-	if ! offline_cpu $cpu; then
-		echo $FUNCNAME $cpu: unexpected fail >&2
-	elif ! cpu_is_offline $cpu; then
-		echo $FUNCNAME $cpu: unexpected offline >&2
-	fi
-}
-
-offline_cpu_expect_fail()
-{
-	local cpu=$1
-
-	if offline_cpu $cpu 2> /dev/null; then
-		echo $FUNCNAME $cpu: unexpected success >&2
-	elif ! cpu_is_online $cpu; then
-		echo $FUNCNAME $cpu: unexpected offline >&2
-	fi
-}
-
-error=-12
-allcpus=0
-priority=0
-online_cpus=0
-online_max=0
-offline_cpus=0
-offline_max=0
-
-while getopts e:ahp: opt; do
-	case $opt in
-	e)
-		error=$OPTARG
-		;;
-	a)
-		allcpus=1
-		;;
-	h)
-		echo "Usage $0 [ -a ] [ -e errno ] [ -p notifier-priority ]"
-		echo -e "\t default offline one cpu"
-		echo -e "\t run with -a option to offline all cpus"
-		exit
-		;;
-	p)
-		priority=$OPTARG
-		;;
-	esac
-done
-
-if ! [ "$error" -ge -4095 -a "$error" -lt 0 ]; then
-	echo "error code must be -4095 <= errno < 0" >&2
-	exit 1
-fi
-
-prerequisite
-
-#
-# Safe test (default) - offline and online one cpu
-#
-if [ $allcpus -eq 0 ]; then
-	echo "Limited scope test: one hotplug cpu"
-	echo -e "\t (leaves cpu in the original state):"
-	echo -e "\t online to offline to online: cpu $online_max"
-	offline_cpu_expect_success $online_max
-	online_cpu_expect_success $online_max
-
-	if [[ $offline_cpus -gt 0 ]]; then
-		echo -e "\t offline to online to offline: cpu $offline_max"
-		online_cpu_expect_success $offline_max
-		offline_cpu_expect_success $offline_max
-	fi
-	exit 0
-else
-	echo "Full scope test: all hotplug cpus"
-	echo -e "\t online all offline cpus"
-	echo -e "\t offline all online cpus"
-	echo -e "\t online all offline cpus"
-fi
-
-#
-# Online all hot-pluggable CPUs
-#
-for cpu in `hotplaggable_offline_cpus`; do
-	online_cpu_expect_success $cpu
-done
-
-#
-# Offline all hot-pluggable CPUs
-#
-for cpu in `hotpluggable_online_cpus`; do
-	offline_cpu_expect_success $cpu
-done
-
-#
-# Online all hot-pluggable CPUs again
-#
-for cpu in `hotplaggable_offline_cpus`; do
-	online_cpu_expect_success $cpu
-done
-
-#
-# Test with cpu notifier error injection
-#
-
-DEBUGFS=`mount -t debugfs | head -1 | awk '{ print $3 }'`
-NOTIFIER_ERR_INJECT_DIR=$DEBUGFS/notifier-error-inject/cpu
-
-prerequisite_extra()
-{
-	msg="skip extra tests:"
-
-	/sbin/modprobe -q -r cpu-notifier-error-inject
-	/sbin/modprobe -q cpu-notifier-error-inject priority=$priority
-
-	if [ ! -d "$DEBUGFS" ]; then
-		echo $msg debugfs is not mounted >&2
-		exit 0
-	fi
-
-	if [ ! -d $NOTIFIER_ERR_INJECT_DIR ]; then
-		echo $msg cpu-notifier-error-inject module is not available >&2
-		exit 0
-	fi
-}
-
-prerequisite_extra
-
-#
-# Offline all hot-pluggable CPUs
-#
-echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_DOWN_PREPARE/error
-for cpu in `hotpluggable_online_cpus`; do
-	offline_cpu_expect_success $cpu
-done
-
-#
-# Test CPU hot-add error handling (offline => online)
-#
-echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_UP_PREPARE/error
-for cpu in `hotplaggable_offline_cpus`; do
-	online_cpu_expect_fail $cpu
-done
-
-#
-# Online all hot-pluggable CPUs
-#
-echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_UP_PREPARE/error
-for cpu in `hotplaggable_offline_cpus`; do
-	online_cpu_expect_success $cpu
-done
-
-#
-# Test CPU hot-remove error handling (online => offline)
-#
-echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_DOWN_PREPARE/error
-for cpu in `hotpluggable_online_cpus`; do
-	offline_cpu_expect_fail $cpu
-done
-
-echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/CPU_DOWN_PREPARE/error
-/sbin/modprobe -q -r cpu-notifier-error-inject
-- 
1.9.1

[PATCH v2 06/19] selftests/efivarfs: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/efivarfs/Makefile | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/efivarfs/Makefile b/tools/testing/selftests/efivarfs/Makefile
index 29e8c6b..d34c4c2 100644
--- a/tools/testing/selftests/efivarfs/Makefile
+++ b/tools/testing/selftests/efivarfs/Makefile
@@ -3,10 +3,25 @@ CFLAGS = -Wall
 
 test_objs = open-unlink create-read
 
-all: $(test_objs)
+TEST_STR = /bin/bash ./efivarfs.sh || echo efivarfs selftests: [FAIL]
+
+all:
+	gcc open-unlink.c -o open-unlink
+	gcc create-read.c -o create-read
+
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./efivarfs.sh $(test_objs) $(INSTALL_KSFT_PATH)
+	@echo echo Start efivarfs test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End efivarfs test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
 
 run_tests: all
-	@/bin/bash ./efivarfs.sh || echo "efivarfs selftests: [FAIL]"
+	@$(TEST_STR)
 
 clean:
 	rm -f $(test_objs)
-- 
1.9.1

[PATCH v2 07/19] selftests/firmware: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/firmware/Makefile | 49 ++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 17 deletions(-)

diff --git a/tools/testing/selftests/firmware/Makefile b/tools/testing/selftests/firmware/Makefile
index e23cce0..2286dbb 100644
--- a/tools/testing/selftests/firmware/Makefile
+++ b/tools/testing/selftests/firmware/Makefile
@@ -1,25 +1,40 @@
 # Makefile for firmware loading selftests
 
 # No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
+
+__fw_filesystem:
+fw_filesystem  = if /bin/sh ./fw_filesystem.sh ; then
+fw_filesystem += echo fw_filesystem: ok;
+fw_filesystem += else echo fw_filesystem: [FAIL];
+fw_filesystem += fi
+
+__fw_userhelper:
+fw_userhelper  = if /bin/sh ./fw_userhelper.sh ; then
+fw_userhelper += echo fw_userhelper: ok;
+fw_userhelper += else
+fw_userhelper += echo fw_userhelper: [FAIL];
+fw_userhelper += fi
+
 all:
 
-fw_filesystem:
-	@if /bin/sh ./fw_filesystem.sh ; then \
-                echo "fw_filesystem: ok"; \
-        else \
-                echo "fw_filesystem: [FAIL]"; \
-                exit 1; \
-        fi
-
-fw_userhelper:
-	@if /bin/sh ./fw_userhelper.sh ; then \
-                echo "fw_userhelper: ok"; \
-        else \
-                echo "fw_userhelper: [FAIL]"; \
-                exit 1; \
-        fi
-
-run_tests: all fw_filesystem fw_userhelper
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./fw_filesystem.sh ./fw_userhelper.sh $(INSTALL_KSFT_PATH)
+	@echo echo Start firmware filesystem test .... >> $(KSELFTEST)
+	@echo "$(fw_filesystem)" >> $(KSELFTEST)
+	@echo echo End firmware filesystem test .... >> $(KSELFTEST)
+	@echo echo -------------------- >> $(KSELFTEST)
+	@echo echo Start firmware userhelper test .... >> $(KSELFTEST)
+	@echo "$(fw_userhelper)" >> $(KSELFTEST)
+	@echo echo End firmware userhelper test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
+run_tests:
+	@$(fw_filesystem)
+	@$(fw_userhelper)
 
 # Nothing to clean up.
 clean:
-- 
1.9.1

[PATCH v2 08/19] selftests/ipc: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/ipc/Makefile | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ipc/Makefile b/tools/testing/selftests/ipc/Makefile
index 74bbefd..cd70f7c 100644
--- a/tools/testing/selftests/ipc/Makefile
+++ b/tools/testing/selftests/ipc/Makefile
@@ -11,6 +11,8 @@ endif
 
 CFLAGS += -I../../../../usr/include/
 
+TEST_STR = ./msgque_test || echo ipc msgque test: [FAIL]
+
 all:
 ifeq ($(ARCH),x86)
 	gcc $(CFLAGS) msgque.c -o msgque_test
@@ -18,8 +20,26 @@ else
 	echo "Not an x86 target, can't build msgque selftest"
 endif
 
+install:
+ifdef INSTALL_KSFT_PATH
+ifeq ($(ARCH),x86)
+	make all
+	install ./msgque_test $(INSTALL_KSFT_PATH)
+	@echo echo Start ipc msgque test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End msgque test .... >> $(KSELFTEST)
+	@echo "echo ============================== >> $(KSELFTEST)
+else
+	@echo Not an x86 target, unable to install ipc msgque selftests
+endif
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	./msgque_test
+ifeq ($(ARCH),x86)
+	@$(TEST_STR)
+endif
 
 clean:
 	rm -fr ./msgque_test
-- 
1.9.1

[PATCH v2 09/19] selftests/kcmp: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/kcmp/Makefile | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kcmp/Makefile b/tools/testing/selftests/kcmp/Makefile
index 8aabd82..753890f 100644
--- a/tools/testing/selftests/kcmp/Makefile
+++ b/tools/testing/selftests/kcmp/Makefile
@@ -14,6 +14,8 @@ CFLAGS += -I../../../../include/
 CFLAGS += -I../../../../usr/include/
 CFLAGS += -I../../../../arch/x86/include/
 
+TEST_STR = ./kcmp_test || echo kcmp_test: [FAIL]
+
 all:
 ifeq ($(ARCH),x86)
 	gcc $(CFLAGS) kcmp_test.c -o kcmp_test
@@ -21,8 +23,26 @@ else
 	echo "Not an x86 target, can't build kcmp selftest"
 endif
 
+install:
+ifdef INSTALL_KSFT_PATH
+ifeq ($(ARCH),x86)
+	install ./kcmp_test $(INSTALL_KSFT_PATH)
+	@echo echo Start kcmp test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo rm -f kcmp-test-file >> $(KSELFTEST)
+	@echo echo End kcmp test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Not an x86 target, unable install kcmp selftests
+endif
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	@./kcmp_test || echo "kcmp_test: [FAIL]"
+ifeq ($(ARCH),x86)
+	@$(TEST_STR)
+endif
 
 clean:
 	$(RM) kcmp_test kcmp-test-file
-- 
1.9.1

[PATCH v2 10/19] selftests/memfd: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/memfd/Makefile | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/memfd/Makefile b/tools/testing/selftests/memfd/Makefile
index b80cd10..a9e06cc 100644
--- a/tools/testing/selftests/memfd/Makefile
+++ b/tools/testing/selftests/memfd/Makefile
@@ -2,19 +2,33 @@ CFLAGS += -D_FILE_OFFSET_BITS=64
 CFLAGS += -I../../../../include/uapi/
 CFLAGS += -I../../../../include/
 
+INSTALL_PROGS = memfd_test fuse_test run_fuse_test.sh
+MEMFD_TEST_STR = ./memfd_test || echo memfd_test: [FAIL]
+FUSE_TEST_STR = ./run_fuse_test.sh || echo fuse_test: [FAIL]
+
 all:
 	gcc $(CFLAGS) memfd_test.c -o memfd_test
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install $(INSTALL_PROGS) $(INSTALL_KSFT_PATH)
+	@echo echo Start memfd test .... >> $(KSELFTEST)
+	@echo "$(MEMFD_TEST_STR)" >> $(KSELFTEST)
+	@echo echo End memfd test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	gcc $(CFLAGS) memfd_test.c -o memfd_test
-	@./memfd_test || echo "memfd_test: [FAIL]"
+	@$(MEMFD_TEST_STR)
 
 build_fuse:
 	gcc $(CFLAGS) fuse_mnt.c `pkg-config fuse --cflags --libs` -o fuse_mnt
 	gcc $(CFLAGS) fuse_test.c -o fuse_test
 
 run_fuse: build_fuse
-	@./run_fuse_test.sh || echo "fuse_test: [FAIL]"
+	@$(FUSE_TEST_STR)
 
 clean:
 	$(RM) memfd_test fuse_test
-- 
1.9.1

[PATCH v2 11/19] selftests/memory-hotplug: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/memory-hotplug/Makefile    |  17 +-
 .../selftests/memory-hotplug/mem-on-off-test.sh    | 238 +++++++++++++++++++++
 .../selftests/memory-hotplug/on-off-test.sh        | 238 ---------------------
 3 files changed, 253 insertions(+), 240 deletions(-)
 create mode 100644 tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
 delete mode 100644 tools/testing/selftests/memory-hotplug/on-off-test.sh

diff --git a/tools/testing/selftests/memory-hotplug/Makefile b/tools/testing/selftests/memory-hotplug/Makefile
index d46b8d4..8921631 100644
--- a/tools/testing/selftests/memory-hotplug/Makefile
+++ b/tools/testing/selftests/memory-hotplug/Makefile
@@ -1,9 +1,22 @@
+TEST_STR=/bin/bash ./mem-on-off-test.sh -r 2 || echo memory-hotplug selftests: [FAIL]
+
 all:
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./mem-on-off-test.sh $(INSTALL_KSFT_PATH)/mem-on-off-test.sh
+	@echo echo Start memory hotplug test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST) >> $(KSELFTEST)
+	@echo echo End memory hotplug test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests:
-	@/bin/bash ./on-off-test.sh -r 2 || echo "memory-hotplug selftests: [FAIL]"
+	@$(TEST_STR)
 
 run_full_test:
-	@/bin/bash ./on-off-test.sh || echo "memory-hotplug selftests: [FAIL]"
+	@/bin/bash ./mem-on-off-test.sh || echo "memory-hotplug selftests: [FAIL]"
 
 clean:
diff --git a/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh b/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
new file mode 100644
index 0000000..6cddde0
--- /dev/null
+++ b/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
@@ -0,0 +1,238 @@
+#!/bin/bash
+
+SYSFS=
+
+prerequisite()
+{
+	msg="skip all tests:"
+
+	if [ $UID != 0 ]; then
+		echo $msg must be run as root >&2
+		exit 0
+	fi
+
+	SYSFS=`mount -t sysfs | head -1 | awk '{ print $3 }'`
+
+	if [ ! -d "$SYSFS" ]; then
+		echo $msg sysfs is not mounted >&2
+		exit 0
+	fi
+
+	if ! ls $SYSFS/devices/system/memory/memory* > /dev/null 2>&1; then
+		echo $msg memory hotplug is not supported >&2
+		exit 0
+	fi
+}
+
+#
+# list all hot-pluggable memory
+#
+hotpluggable_memory()
+{
+	local state=${1:-.\*}
+
+	for memory in $SYSFS/devices/system/memory/memory*; do
+		if grep -q 1 $memory/removable &&
+		   grep -q $state $memory/state; then
+			echo ${memory##/*/memory}
+		fi
+	done
+}
+
+hotplaggable_offline_memory()
+{
+	hotpluggable_memory offline
+}
+
+hotpluggable_online_memory()
+{
+	hotpluggable_memory online
+}
+
+memory_is_online()
+{
+	grep -q online $SYSFS/devices/system/memory/memory$1/state
+}
+
+memory_is_offline()
+{
+	grep -q offline $SYSFS/devices/system/memory/memory$1/state
+}
+
+online_memory()
+{
+	echo online > $SYSFS/devices/system/memory/memory$1/state
+}
+
+offline_memory()
+{
+	echo offline > $SYSFS/devices/system/memory/memory$1/state
+}
+
+online_memory_expect_success()
+{
+	local memory=$1
+
+	if ! online_memory $memory; then
+		echo $FUNCNAME $memory: unexpected fail >&2
+	elif ! memory_is_online $memory; then
+		echo $FUNCNAME $memory: unexpected offline >&2
+	fi
+}
+
+online_memory_expect_fail()
+{
+	local memory=$1
+
+	if online_memory $memory 2> /dev/null; then
+		echo $FUNCNAME $memory: unexpected success >&2
+	elif ! memory_is_offline $memory; then
+		echo $FUNCNAME $memory: unexpected online >&2
+	fi
+}
+
+offline_memory_expect_success()
+{
+	local memory=$1
+
+	if ! offline_memory $memory; then
+		echo $FUNCNAME $memory: unexpected fail >&2
+	elif ! memory_is_offline $memory; then
+		echo $FUNCNAME $memory: unexpected offline >&2
+	fi
+}
+
+offline_memory_expect_fail()
+{
+	local memory=$1
+
+	if offline_memory $memory 2> /dev/null; then
+		echo $FUNCNAME $memory: unexpected success >&2
+	elif ! memory_is_online $memory; then
+		echo $FUNCNAME $memory: unexpected offline >&2
+	fi
+}
+
+error=-12
+priority=0
+ratio=10
+
+while getopts e:hp:r: opt; do
+	case $opt in
+	e)
+		error=$OPTARG
+		;;
+	h)
+		echo "Usage $0 [ -e errno ] [ -p notifier-priority ] [ -r percent-of-memory-to-offline ]"
+		exit
+		;;
+	p)
+		priority=$OPTARG
+		;;
+	r)
+		ratio=$OPTARG
+		;;
+	esac
+done
+
+if ! [ "$error" -ge -4095 -a "$error" -lt 0 ]; then
+	echo "error code must be -4095 <= errno < 0" >&2
+	exit 1
+fi
+
+prerequisite
+
+echo "Test scope: $ratio% hotplug memory"
+echo -e "\t online all hotplug memory in offline state"
+echo -e "\t offline $ratio% hotplug memory in online state"
+echo -e "\t online all hotplug memory in offline state"
+
+#
+# Online all hot-pluggable memory
+#
+for memory in `hotplaggable_offline_memory`; do
+	echo offline-online $memory
+	online_memory_expect_success $memory
+done
+
+#
+# Offline $ratio percent of hot-pluggable memory
+#
+for memory in `hotpluggable_online_memory`; do
+	if [ $((RANDOM % 100)) -lt $ratio ]; then
+		echo online-offline $memory
+		offline_memory_expect_success $memory
+	fi
+done
+
+#
+# Online all hot-pluggable memory again
+#
+for memory in `hotplaggable_offline_memory`; do
+	echo offline-online $memory
+	online_memory_expect_success $memory
+done
+
+#
+# Test with memory notifier error injection
+#
+
+DEBUGFS=`mount -t debugfs | head -1 | awk '{ print $3 }'`
+NOTIFIER_ERR_INJECT_DIR=$DEBUGFS/notifier-error-inject/memory
+
+prerequisite_extra()
+{
+	msg="skip extra tests:"
+
+	/sbin/modprobe -q -r memory-notifier-error-inject
+	/sbin/modprobe -q memory-notifier-error-inject priority=$priority
+
+	if [ ! -d "$DEBUGFS" ]; then
+		echo $msg debugfs is not mounted >&2
+		exit 0
+	fi
+
+	if [ ! -d $NOTIFIER_ERR_INJECT_DIR ]; then
+		echo $msg memory-notifier-error-inject module is not available >&2
+		exit 0
+	fi
+}
+
+prerequisite_extra
+
+#
+# Offline $ratio percent of hot-pluggable memory
+#
+echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_OFFLINE/error
+for memory in `hotpluggable_online_memory`; do
+	if [ $((RANDOM % 100)) -lt $ratio ]; then
+		offline_memory_expect_success $memory
+	fi
+done
+
+#
+# Test memory hot-add error handling (offline => online)
+#
+echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_ONLINE/error
+for memory in `hotplaggable_offline_memory`; do
+	online_memory_expect_fail $memory
+done
+
+#
+# Online all hot-pluggable memory
+#
+echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_ONLINE/error
+for memory in `hotplaggable_offline_memory`; do
+	online_memory_expect_success $memory
+done
+
+#
+# Test memory hot-remove error handling (online => offline)
+#
+echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_OFFLINE/error
+for memory in `hotpluggable_online_memory`; do
+	offline_memory_expect_fail $memory
+done
+
+echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_OFFLINE/error
+/sbin/modprobe -q -r memory-notifier-error-inject
diff --git a/tools/testing/selftests/memory-hotplug/on-off-test.sh b/tools/testing/selftests/memory-hotplug/on-off-test.sh
deleted file mode 100644
index 6cddde0..0000000
--- a/tools/testing/selftests/memory-hotplug/on-off-test.sh
+++ /dev/null
@@ -1,238 +0,0 @@
-#!/bin/bash
-
-SYSFS=
-
-prerequisite()
-{
-	msg="skip all tests:"
-
-	if [ $UID != 0 ]; then
-		echo $msg must be run as root >&2
-		exit 0
-	fi
-
-	SYSFS=`mount -t sysfs | head -1 | awk '{ print $3 }'`
-
-	if [ ! -d "$SYSFS" ]; then
-		echo $msg sysfs is not mounted >&2
-		exit 0
-	fi
-
-	if ! ls $SYSFS/devices/system/memory/memory* > /dev/null 2>&1; then
-		echo $msg memory hotplug is not supported >&2
-		exit 0
-	fi
-}
-
-#
-# list all hot-pluggable memory
-#
-hotpluggable_memory()
-{
-	local state=${1:-.\*}
-
-	for memory in $SYSFS/devices/system/memory/memory*; do
-		if grep -q 1 $memory/removable &&
-		   grep -q $state $memory/state; then
-			echo ${memory##/*/memory}
-		fi
-	done
-}
-
-hotplaggable_offline_memory()
-{
-	hotpluggable_memory offline
-}
-
-hotpluggable_online_memory()
-{
-	hotpluggable_memory online
-}
-
-memory_is_online()
-{
-	grep -q online $SYSFS/devices/system/memory/memory$1/state
-}
-
-memory_is_offline()
-{
-	grep -q offline $SYSFS/devices/system/memory/memory$1/state
-}
-
-online_memory()
-{
-	echo online > $SYSFS/devices/system/memory/memory$1/state
-}
-
-offline_memory()
-{
-	echo offline > $SYSFS/devices/system/memory/memory$1/state
-}
-
-online_memory_expect_success()
-{
-	local memory=$1
-
-	if ! online_memory $memory; then
-		echo $FUNCNAME $memory: unexpected fail >&2
-	elif ! memory_is_online $memory; then
-		echo $FUNCNAME $memory: unexpected offline >&2
-	fi
-}
-
-online_memory_expect_fail()
-{
-	local memory=$1
-
-	if online_memory $memory 2> /dev/null; then
-		echo $FUNCNAME $memory: unexpected success >&2
-	elif ! memory_is_offline $memory; then
-		echo $FUNCNAME $memory: unexpected online >&2
-	fi
-}
-
-offline_memory_expect_success()
-{
-	local memory=$1
-
-	if ! offline_memory $memory; then
-		echo $FUNCNAME $memory: unexpected fail >&2
-	elif ! memory_is_offline $memory; then
-		echo $FUNCNAME $memory: unexpected offline >&2
-	fi
-}
-
-offline_memory_expect_fail()
-{
-	local memory=$1
-
-	if offline_memory $memory 2> /dev/null; then
-		echo $FUNCNAME $memory: unexpected success >&2
-	elif ! memory_is_online $memory; then
-		echo $FUNCNAME $memory: unexpected offline >&2
-	fi
-}
-
-error=-12
-priority=0
-ratio=10
-
-while getopts e:hp:r: opt; do
-	case $opt in
-	e)
-		error=$OPTARG
-		;;
-	h)
-		echo "Usage $0 [ -e errno ] [ -p notifier-priority ] [ -r percent-of-memory-to-offline ]"
-		exit
-		;;
-	p)
-		priority=$OPTARG
-		;;
-	r)
-		ratio=$OPTARG
-		;;
-	esac
-done
-
-if ! [ "$error" -ge -4095 -a "$error" -lt 0 ]; then
-	echo "error code must be -4095 <= errno < 0" >&2
-	exit 1
-fi
-
-prerequisite
-
-echo "Test scope: $ratio% hotplug memory"
-echo -e "\t online all hotplug memory in offline state"
-echo -e "\t offline $ratio% hotplug memory in online state"
-echo -e "\t online all hotplug memory in offline state"
-
-#
-# Online all hot-pluggable memory
-#
-for memory in `hotplaggable_offline_memory`; do
-	echo offline-online $memory
-	online_memory_expect_success $memory
-done
-
-#
-# Offline $ratio percent of hot-pluggable memory
-#
-for memory in `hotpluggable_online_memory`; do
-	if [ $((RANDOM % 100)) -lt $ratio ]; then
-		echo online-offline $memory
-		offline_memory_expect_success $memory
-	fi
-done
-
-#
-# Online all hot-pluggable memory again
-#
-for memory in `hotplaggable_offline_memory`; do
-	echo offline-online $memory
-	online_memory_expect_success $memory
-done
-
-#
-# Test with memory notifier error injection
-#
-
-DEBUGFS=`mount -t debugfs | head -1 | awk '{ print $3 }'`
-NOTIFIER_ERR_INJECT_DIR=$DEBUGFS/notifier-error-inject/memory
-
-prerequisite_extra()
-{
-	msg="skip extra tests:"
-
-	/sbin/modprobe -q -r memory-notifier-error-inject
-	/sbin/modprobe -q memory-notifier-error-inject priority=$priority
-
-	if [ ! -d "$DEBUGFS" ]; then
-		echo $msg debugfs is not mounted >&2
-		exit 0
-	fi
-
-	if [ ! -d $NOTIFIER_ERR_INJECT_DIR ]; then
-		echo $msg memory-notifier-error-inject module is not available >&2
-		exit 0
-	fi
-}
-
-prerequisite_extra
-
-#
-# Offline $ratio percent of hot-pluggable memory
-#
-echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_OFFLINE/error
-for memory in `hotpluggable_online_memory`; do
-	if [ $((RANDOM % 100)) -lt $ratio ]; then
-		offline_memory_expect_success $memory
-	fi
-done
-
-#
-# Test memory hot-add error handling (offline => online)
-#
-echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_ONLINE/error
-for memory in `hotplaggable_offline_memory`; do
-	online_memory_expect_fail $memory
-done
-
-#
-# Online all hot-pluggable memory
-#
-echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_ONLINE/error
-for memory in `hotplaggable_offline_memory`; do
-	online_memory_expect_success $memory
-done
-
-#
-# Test memory hot-remove error handling (online => offline)
-#
-echo $error > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_OFFLINE/error
-for memory in `hotpluggable_online_memory`; do
-	offline_memory_expect_fail $memory
-done
-
-echo 0 > $NOTIFIER_ERR_INJECT_DIR/actions/MEM_GOING_OFFLINE/error
-/sbin/modprobe -q -r memory-notifier-error-inject
-- 
1.9.1

[PATCH v2 12/19] selftests/mount: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/mount/Makefile | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/mount/Makefile b/tools/testing/selftests/mount/Makefile
index 337d853..7533097 100644
--- a/tools/testing/selftests/mount/Makefile
+++ b/tools/testing/selftests/mount/Makefile
@@ -1,5 +1,7 @@
 # Makefile for mount selftests.
 
+TEST_STR = if [ -f /proc/self/uid_map ] ; then ./unprivileged-remount-test ; fi
+
 all: unprivileged-remount-test
 
 unprivileged-remount-test: unprivileged-remount-test.c
@@ -7,7 +9,18 @@ unprivileged-remount-test: unprivileged-remount-test.c
 
 # Allow specific tests to be selected.
 test_unprivileged_remount: unprivileged-remount-test
-	@if [ -f /proc/self/uid_map ] ; then ./unprivileged-remount-test ; fi
+	@$(TEST_STR)
+
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./unprivileged-remount-test $(INSTALL_KSFT_PATH)
+	@echo echo Start mount test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End mount test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
 
 run_tests: all test_unprivileged_remount
 
-- 
1.9.1

[PATCH v2 13/19] selftests/mqueue: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/mqueue/Makefile | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/mqueue/Makefile b/tools/testing/selftests/mqueue/Makefile
index 8056e2e..7a7e5d3 100644
--- a/tools/testing/selftests/mqueue/Makefile
+++ b/tools/testing/selftests/mqueue/Makefile
@@ -1,10 +1,24 @@
+MQ_OPEN_TEST_STR = ./mq_open_tests /test1 || echo mq_open_tests: [FAIL]
+MQ_PERF_TEST_STR = ./mq_perf_tests || echo mq_perf_tests: [FAIL]
+
 all:
 	gcc -O2 mq_open_tests.c -o mq_open_tests -lrt
 	gcc -O2 -o mq_perf_tests mq_perf_tests.c -lrt -lpthread -lpopt
 
-run_tests:
-	@./mq_open_tests /test1 || echo "mq_open_tests: [FAIL]"
-	@./mq_perf_tests || echo "mq_perf_tests: [FAIL]"
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./mq_open_tests ./mq_perf_tests $(INSTALL_KSFT_PATH)
+	@echo echo Start mqueue test .... >> $(KSELFTEST)
+	@echo "$(MQ_OPEN_TEST_STR)" >> $(KSELFTEST)
+	@echo "$(MQ_PERF_TEST_STR)" >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
+run_tests: all
+	@$(MQ_OPEN_TEST_STR)
+	@$(MQ_PERF_TEST_STR)
 
 clean:
 	rm -f mq_open_tests mq_perf_tests
-- 
1.9.1

[PATCH v2 14/19] selftests/net: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/net/Makefile | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 62f22cc..16550eb 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -6,14 +6,31 @@ CFLAGS = -Wall -O2 -g
 CFLAGS += -I../../../../usr/include/
 
 NET_PROGS = socket psock_fanout psock_tpacket
+INSTALL_PROGS = run_netsocktests run_afpackettests test_bpf.sh $(NET_PROGS)
+NETSOCK_TEST_STR = /bin/sh ./run_netsocktests || echo sockettests: [FAIL]
+AFPKT_TEST_STR = /bin/sh ./run_afpackettests || echo afpackettests: [FAIL]
+BFP_TEST_STR = ./test_bpf.sh
 
 all: $(NET_PROGS)
 %: %.c
 	$(CC) $(CFLAGS) -o $@ $^
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install $(INSTALL_PROGS) $(INSTALL_KSFT_PATH)
+	@echo echo Start net test .... >> $(KSELFTEST)
+	@echo "$(NETSOCK_TEST_STR)" >> $(KSELFTEST)
+	@echo "$(AFPKT_TEST_STR)" >> $(KSELFTEST)
+	@echo "$(BFP_TEST_STR)" >> $(KSELFTEST)
+	@echo echo End net test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	@/bin/sh ./run_netsocktests || echo "sockettests: [FAIL]"
-	@/bin/sh ./run_afpackettests || echo "afpackettests: [FAIL]"
-	./test_bpf.sh
+	@$(NETSOCK_TEST_STR)
+	@$(AFPKT_TEST_STR)
+	@$(BFP_TEST_STR)
 clean:
 	$(RM) $(NET_PROGS)
-- 
1.9.1

[PATCH v2 15/19] selftests/ptrace: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/ptrace/Makefile | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/ptrace/Makefile b/tools/testing/selftests/ptrace/Makefile
index 47ae2d3..9ef3c4b 100644
--- a/tools/testing/selftests/ptrace/Makefile
+++ b/tools/testing/selftests/ptrace/Makefile
@@ -1,10 +1,23 @@
 CFLAGS += -iquote../../../../include/uapi -Wall
-peeksiginfo: peeksiginfo.c
 
-all: peeksiginfo
+TEST_STR = ./peeksiginfo || echo peeksiginfo selftests: [FAIL]
+
+all:
+	gcc peeksiginfo.c -o peeksiginfo
+
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./peeksiginfo $(INSTALL_KSFT_PATH)
+	@echo echo Start ptrace test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End ptrace test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
 
 clean:
 	rm -f peeksiginfo
 
 run_tests: all
-	@./peeksiginfo || echo "peeksiginfo selftests: [FAIL]"
+	@$(TEST_STR)
-- 
1.9.1

[PATCH v2 15/19] selftests/ptrace: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org>
---
 tools/testing/selftests/ptrace/Makefile | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/ptrace/Makefile b/tools/testing/selftests/ptrace/Makefile
index 47ae2d3..9ef3c4b 100644
--- a/tools/testing/selftests/ptrace/Makefile
+++ b/tools/testing/selftests/ptrace/Makefile
@@ -1,10 +1,23 @@
 CFLAGS += -iquote../../../../include/uapi -Wall
-peeksiginfo: peeksiginfo.c
 
-all: peeksiginfo
+TEST_STR = ./peeksiginfo || echo peeksiginfo selftests: [FAIL]
+
+all:
+	gcc peeksiginfo.c -o peeksiginfo
+
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./peeksiginfo $(INSTALL_KSFT_PATH)
+	@echo echo Start ptrace test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End ptrace test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
 
 clean:
 	rm -f peeksiginfo
 
 run_tests: all
-	@./peeksiginfo || echo "peeksiginfo selftests: [FAIL]"
+	@$(TEST_STR)
-- 
1.9.1

[PATCH v2 16/19] selftests/sysctl: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/sysctl/Makefile | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/sysctl/Makefile b/tools/testing/selftests/sysctl/Makefile
index 0a92ada..34cca1b 100644
--- a/tools/testing/selftests/sysctl/Makefile
+++ b/tools/testing/selftests/sysctl/Makefile
@@ -4,12 +4,31 @@
 # No binaries, but make sure arg-less "make" doesn't trigger "run_tests".
 all:
 
+INSTALL_PROGS = common_tests run_numerictests run_stringtests
+NUMERIC_TEST_STR = /bin/sh ./run_numerictests
+STRING_TEST_STR = /bin/sh ./run_stringtests
+
 # Allow specific tests to be selected.
 test_num:
-	@/bin/sh ./run_numerictests
+	@$(NUMERIC_TEST_STR)
 
 test_string:
-	@/bin/sh ./run_stringtests
+	@$(STRING_TEST_STR)
+
+install: all
+ifdef INSTALL_KSFT_PATH
+	install $(INSTALL_PROGS) $(INSTALL_KSFT_PATH)
+	@echo echo Start sysctl numeric tests .... >> $(KSELFTEST)
+	@echo "$(NUMERIC_TEST_STR)" >> $(KSELFTEST)
+	@echo echo End sysctl numeric tests .... >> $(KSELFTEST)
+	@echo echo -------------------- >> $(KSELFTEST)
+	@echo echo Start sysctl string tests .... >> $(KSELFTEST)
+	@echo "$(STRING_TEST_STR)" >> $(KSELFTEST)
+	@echo echo End sysctl string test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
 
 run_tests: all test_num test_string
 
-- 
1.9.1

[PATCH v2 17/19] selftests/timers: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/timers/Makefile | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/timers/Makefile b/tools/testing/selftests/timers/Makefile
index eb2859f..0c3c67b 100644
--- a/tools/testing/selftests/timers/Makefile
+++ b/tools/testing/selftests/timers/Makefile
@@ -1,8 +1,21 @@
+TEST_STR = ./posix_timers
+
 all:
 	gcc posix_timers.c -o posix_timers -lrt
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./posix_timers $(INSTALL_KSFT_PATH)
+	@echo echo Start timers test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End timers test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	./posix_timers
+	@$(TEST_STR)
 
 clean:
 	rm -f ./posix_timers
-- 
1.9.1

[PATCH v2 18/19] selftests/vm: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/vm/Makefile | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile
index 4c4b1f6..7fbe551 100644
--- a/tools/testing/selftests/vm/Makefile
+++ b/tools/testing/selftests/vm/Makefile
@@ -4,13 +4,25 @@ CC = $(CROSS_COMPILE)gcc
 CFLAGS = -Wall
 BINARIES = hugepage-mmap hugepage-shm map_hugetlb thuge-gen hugetlbfstest
 BINARIES += transhuge-stress
+TEST_STR = /bin/sh ./run_vmtests || echo vmtests: [FAIL]
 
 all: $(BINARIES)
 %: %.c
 	$(CC) $(CFLAGS) -o $@ $^
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install run_vmtests $(BINARIES) $(INSTALL_KSFT_PATH)
+	@echo echo Start vm test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End vm test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	@/bin/sh ./run_vmtests || (echo "vmtests: [FAIL]"; exit 1)
+	@$(TEST_STR)
 
 clean:
 	$(RM) $(BINARIES)
-- 
1.9.1

[PATCH v2 19/19] selftests/user: add install target to enable installing test

[Shuah Khan]

Add a new make target to enable installing test. This target
installs test in the kselftest install location and add to the
kselftest script to run the test. Install target can be run
only from top level source dir.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/user/Makefile | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/user/Makefile b/tools/testing/selftests/user/Makefile
index 12c9d15..b62962a 100644
--- a/tools/testing/selftests/user/Makefile
+++ b/tools/testing/selftests/user/Makefile
@@ -1,7 +1,20 @@
 # Makefile for user memory selftests
 
+TEST_STR = ./test_user_copy.sh
+
 # No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
 all:
 
+install:
+ifdef INSTALL_KSFT_PATH
+	install ./test_user_copy.sh $(INSTALL_KSFT_PATH)
+	@echo echo Start user copy test .... >> $(KSELFTEST)
+	@echo "$(TEST_STR)" >> $(KSELFTEST)
+	@echo echo End user copy test .... >> $(KSELFTEST)
+	@echo echo ============================== >> $(KSELFTEST)
+else
+	@echo Run make kselftest_install in top level source directory
+endif
+
 run_tests: all
-	./test_user_copy.sh
+	@$(TEST_STR)
-- 
1.9.1

Re: [PATCH v2 07/19] selftests/firmware: add install target to enable installing test

[Kees Cook]

Hi,

Sorry, I still really don't like this approach. While it is all in one
place (thank you for that), I think it isn't a form that is very
workable for the people maintaining the self tests. How about this,
instead of per-Makefile customization, why not define an execution
framework for these tests instead.

For example, how about every test directory must have a Makefile with
the needed binary targets. A common makefile could be included that
defines the "run_tests" target that calls the script "run_tests.sh"
that is a shell script in the current directory. (For inspiration, see
how kernel modules can be built out of tree.)

The "run_tests.sh" scripts could all include a common shell script,
say "../common.sh" that provides any common variables, functions, etc
(e.g. things like "Start $name test ..." should be in common.sh
instead of repeated in every script, the installation logic can be in
once place instead of repeated).

Then along side common.sh could be "run_installed_tests.sh" or
something, used on the installed target, that would traverse each
directory, etc. From this, we can have a much more data-driven
framework, and a common approach to running tests.

As such, we should declare up front how tests should behave on
failure. And the top-level test runner can do things like count the
number of tests, failures, etc.

Then, instead of splitting up the patches by test directory, you can
split them up by logical changes (e.g. defining the common "run_tests"
target, and then removing the target from all the tests by including
the common makefile stub that defines it).

-Kees

On Tue, Nov 11, 2014 at 12:27 PM, Shuah Khan <shuahkh@osg.samsung.com> wrote:
> Add a new make target to enable installing test. This target
> installs test in the kselftest install location and add to the
> kselftest script to run the test. Install target can be run
> only from top level source dir.
>
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
>  tools/testing/selftests/firmware/Makefile | 49 ++++++++++++++++++++-----------
>  1 file changed, 32 insertions(+), 17 deletions(-)
>
> diff --git a/tools/testing/selftests/firmware/Makefile b/tools/testing/selftests/firmware/Makefile
> index e23cce0..2286dbb 100644
> --- a/tools/testing/selftests/firmware/Makefile
> +++ b/tools/testing/selftests/firmware/Makefile
> @@ -1,25 +1,40 @@
>  # Makefile for firmware loading selftests
>
>  # No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
> +
> +__fw_filesystem:
> +fw_filesystem  = if /bin/sh ./fw_filesystem.sh ; then
> +fw_filesystem += echo fw_filesystem: ok;
> +fw_filesystem += else echo fw_filesystem: [FAIL];
> +fw_filesystem += fi
> +
> +__fw_userhelper:
> +fw_userhelper  = if /bin/sh ./fw_userhelper.sh ; then
> +fw_userhelper += echo fw_userhelper: ok;
> +fw_userhelper += else
> +fw_userhelper += echo fw_userhelper: [FAIL];
> +fw_userhelper += fi
> +
>  all:
>
> -fw_filesystem:
> -       @if /bin/sh ./fw_filesystem.sh ; then \
> -                echo "fw_filesystem: ok"; \
> -        else \
> -                echo "fw_filesystem: [FAIL]"; \
> -                exit 1; \
> -        fi
> -
> -fw_userhelper:
> -       @if /bin/sh ./fw_userhelper.sh ; then \
> -                echo "fw_userhelper: ok"; \
> -        else \
> -                echo "fw_userhelper: [FAIL]"; \
> -                exit 1; \
> -        fi
> -
> -run_tests: all fw_filesystem fw_userhelper
> +install:
> +ifdef INSTALL_KSFT_PATH
> +       install ./fw_filesystem.sh ./fw_userhelper.sh $(INSTALL_KSFT_PATH)
> +       @echo echo Start firmware filesystem test .... >> $(KSELFTEST)
> +       @echo "$(fw_filesystem)" >> $(KSELFTEST)
> +       @echo echo End firmware filesystem test .... >> $(KSELFTEST)
> +       @echo echo -------------------- >> $(KSELFTEST)
> +       @echo echo Start firmware userhelper test .... >> $(KSELFTEST)
> +       @echo "$(fw_userhelper)" >> $(KSELFTEST)
> +       @echo echo End firmware userhelper test .... >> $(KSELFTEST)
> +       @echo echo ============================== >> $(KSELFTEST)
> +else
> +       @echo Run make kselftest_install in top level source directory
> +endif
> +
> +run_tests:
> +       @$(fw_filesystem)
> +       @$(fw_userhelper)
>
>  # Nothing to clean up.
>  clean:
> --
> 1.9.1
>



-- 
Kees Cook
Chrome OS Security

Re: [PATCH v2 07/19] selftests/firmware: add install target to enable installing test

[Kees Cook]

Hi,

Sorry, I still really don't like this approach. While it is all in one
place (thank you for that), I think it isn't a form that is very
workable for the people maintaining the self tests. How about this,
instead of per-Makefile customization, why not define an execution
framework for these tests instead.

For example, how about every test directory must have a Makefile with
the needed binary targets. A common makefile could be included that
defines the "run_tests" target that calls the script "run_tests.sh"
that is a shell script in the current directory. (For inspiration, see
how kernel modules can be built out of tree.)

The "run_tests.sh" scripts could all include a common shell script,
say "../common.sh" that provides any common variables, functions, etc
(e.g. things like "Start $name test ..." should be in common.sh
instead of repeated in every script, the installation logic can be in
once place instead of repeated).

Then along side common.sh could be "run_installed_tests.sh" or
something, used on the installed target, that would traverse each
directory, etc. From this, we can have a much more data-driven
framework, and a common approach to running tests.

As such, we should declare up front how tests should behave on
failure. And the top-level test runner can do things like count the
number of tests, failures, etc.

Then, instead of splitting up the patches by test directory, you can
split them up by logical changes (e.g. defining the common "run_tests"
target, and then removing the target from all the tests by including
the common makefile stub that defines it).

-Kees

On Tue, Nov 11, 2014 at 12:27 PM, Shuah Khan <shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org> wrote:
> Add a new make target to enable installing test. This target
> installs test in the kselftest install location and add to the
> kselftest script to run the test. Install target can be run
> only from top level source dir.
>
> Signed-off-by: Shuah Khan <shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org>
> ---
>  tools/testing/selftests/firmware/Makefile | 49 ++++++++++++++++++++-----------
>  1 file changed, 32 insertions(+), 17 deletions(-)
>
> diff --git a/tools/testing/selftests/firmware/Makefile b/tools/testing/selftests/firmware/Makefile
> index e23cce0..2286dbb 100644
> --- a/tools/testing/selftests/firmware/Makefile
> +++ b/tools/testing/selftests/firmware/Makefile
> @@ -1,25 +1,40 @@
>  # Makefile for firmware loading selftests
>
>  # No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
> +
> +__fw_filesystem:
> +fw_filesystem  = if /bin/sh ./fw_filesystem.sh ; then
> +fw_filesystem += echo fw_filesystem: ok;
> +fw_filesystem += else echo fw_filesystem: [FAIL];
> +fw_filesystem += fi
> +
> +__fw_userhelper:
> +fw_userhelper  = if /bin/sh ./fw_userhelper.sh ; then
> +fw_userhelper += echo fw_userhelper: ok;
> +fw_userhelper += else
> +fw_userhelper += echo fw_userhelper: [FAIL];
> +fw_userhelper += fi
> +
>  all:
>
> -fw_filesystem:
> -       @if /bin/sh ./fw_filesystem.sh ; then \
> -                echo "fw_filesystem: ok"; \
> -        else \
> -                echo "fw_filesystem: [FAIL]"; \
> -                exit 1; \
> -        fi
> -
> -fw_userhelper:
> -       @if /bin/sh ./fw_userhelper.sh ; then \
> -                echo "fw_userhelper: ok"; \
> -        else \
> -                echo "fw_userhelper: [FAIL]"; \
> -                exit 1; \
> -        fi
> -
> -run_tests: all fw_filesystem fw_userhelper
> +install:
> +ifdef INSTALL_KSFT_PATH
> +       install ./fw_filesystem.sh ./fw_userhelper.sh $(INSTALL_KSFT_PATH)
> +       @echo echo Start firmware filesystem test .... >> $(KSELFTEST)
> +       @echo "$(fw_filesystem)" >> $(KSELFTEST)
> +       @echo echo End firmware filesystem test .... >> $(KSELFTEST)
> +       @echo echo -------------------- >> $(KSELFTEST)
> +       @echo echo Start firmware userhelper test .... >> $(KSELFTEST)
> +       @echo "$(fw_userhelper)" >> $(KSELFTEST)
> +       @echo echo End firmware userhelper test .... >> $(KSELFTEST)
> +       @echo echo ============================== >> $(KSELFTEST)
> +else
> +       @echo Run make kselftest_install in top level source directory
> +endif
> +
> +run_tests:
> +       @$(fw_filesystem)
> +       @$(fw_userhelper)
>
>  # Nothing to clean up.
>  clean:
> --
> 1.9.1
>



-- 
Kees Cook
Chrome OS Security

Re: [PATCH v2 07/19] selftests/firmware: add install target to enable installing test

[Shuah Khan]

On 11/11/2014 02:29 PM, Kees Cook wrote:
> Hi,
> 
> Sorry, I still really don't like this approach. While it is all in one
> place (thank you for that), I think it isn't a form that is very
> workable for the people maintaining the self tests. How about this,
> instead of per-Makefile customization, why not define an execution
> framework for these tests instead.

If I understand correctly, sounds like you don't like the way
install target is implemented in the individual test Makefiles
and the changes I made to run_tests targets to address the code
duplication concern.

At the moment there is no duplicate code in this patch series
between install and run_tests targets. This is a  first step
towards standardizing the framework and a definite improvement
over what we have currently. As I mentioned earlier, my goal
is to make it easier for developers to install and run the
existing tests and evolve the framework as we go.

Assuming my understanding is correct that:

-- install and run_tests targets in individual tests can be
   refined and automated with a common Makefile approach you
   proposed.
-- the rest of the user-interface kselftest_install and kselftest
   are good.

I would like to propose that we get started with the current
implementation and refine it based on the following ideas you
suggested. The refinements you are recommending are confined
to selftests and can be made after the kselftest_install
gets added. Adding kselftest_install makes it easier to make
the refinements as it defines overall UI.

> 
> For example, how about every test directory must have a Makefile with
> the needed binary targets. A common makefile could be included that
> defines the "run_tests" target that calls the script "run_tests.sh"
> that is a shell script in the current directory. (For inspiration, see
> how kernel modules can be built out of tree.)
> 
> The "run_tests.sh" scripts could all include a common shell script,
> say "../common.sh" that provides any common variables, functions, etc
> (e.g. things like "Start $name test ..." should be in common.sh
> instead of repeated in every script, the installation logic can be in
> once place instead of repeated).
> 
> Then along side common.sh could be "run_installed_tests.sh" or
> something, used on the installed target, that would traverse each
> directory, etc. From this, we can have a much more data-driven
> framework, and a common approach to running tests.
> 
> As such, we should declare up front how tests should behave on
> failure. And the top-level test runner can do things like count the
> number of tests, failures, etc.
> 
> Then, instead of splitting up the patches by test directory, you can
> split them up by logical changes (e.g. defining the common "run_tests"
> target, and then removing the target from all the tests by including
> the common makefile stub that defines it).
> 

These are good ideas and I am with on evolving the framework to make
it easier to maintain individual tests. Patches are welcome.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

On 11/11/2014 01:27 PM, Shuah Khan wrote:
> Add a new make target to install to install kernel selftests.
> This new target will build and install selftests. kselftest
> target now depends on kselftest_install and runs the generated
> kselftest script to reduce duplicate work and for common look
> and feel when running tests.
> 
> Approach:
> 
> make kselftest_target:
> -- exports kselftest INSTALL_KSFT_PATH
>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> -- exports path for ksefltest.sh
> -- runs selftests make install target:
> 
> selftests make install target
> -- creates kselftest.sh script in install install dir
> -- runs install targets for all INSTALL_TARGETS
>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>           to not add more content to patch v1 series. This work
>           will happen soon. In this series these two targets are
>           run after running the generated kselftest script, without
>           any regression in the way these tests are run with
>           "make kselftest" prior to this work.)
> -- install target can be run only from top level source dir.
> 
> Individual test make install targets:
> -- install test programs and/or scripts in install dir
> -- append to the ksefltest.sh file to add commands to run test
> -- install target can be run only from top level source dir.
> 
> Adds the following new ways to initiate selftests:
> -- Installing and running kselftest from install directory
>    by running  "make kselftest"
> -- Running kselftest script from install directory
> 
> Maintains the following ways to run tests:
> -- make -C tools/testing/selftests run_tests
> -- make -C tools/testing/selftests TARGETS=target run_tests
>    Ability specify targets: e.g TARGETS=net
> -- make run_tests from tools/testing/selftests
> -- make run_tests from individual test directories:
>    e.g: make run_tests in tools/testing/selftests/breakpoints
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
>  Makefile | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)

Hi Marek,

Did you get a chance to take a look at this patch?

-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 07/19] selftests/firmware: add install target to enable installing test

[Shuah Khan]

On 11/11/2014 06:06 PM, Shuah Khan wrote:
> On 11/11/2014 02:29 PM, Kees Cook wrote:
>> Hi,
>>
>> Sorry, I still really don't like this approach. While it is all in one
>> place (thank you for that), I think it isn't a form that is very
>> workable for the people maintaining the self tests. How about this,
>> instead of per-Makefile customization, why not define an execution
>> framework for these tests instead.
> 
> If I understand correctly, sounds like you don't like the way
> install target is implemented in the individual test Makefiles
> and the changes I made to run_tests targets to address the code
> duplication concern.
> 
> At the moment there is no duplicate code in this patch series
> between install and run_tests targets. This is a  first step
> towards standardizing the framework and a definite improvement
> over what we have currently. As I mentioned earlier, my goal
> is to make it easier for developers to install and run the
> existing tests and evolve the framework as we go.
> 
> Assuming my understanding is correct that:
> 
> -- install and run_tests targets in individual tests can be
>    refined and automated with a common Makefile approach you
>    proposed.
> -- the rest of the user-interface kselftest_install and kselftest
>    are good.
> 
> I would like to propose that we get started with the current
> implementation and refine it based on the following ideas you
> suggested. The refinements you are recommending are confined
> to selftests and can be made after the kselftest_install
> gets added. Adding kselftest_install makes it easier to make
> the refinements as it defines overall UI.
> 

Hi Kees,

Are you ok with the above proposal? I understand this approach
might not be perfect, however it is a step in the right direction
to enable and make it easier to run them. I would like to get some
initial work in for 3.19 if at all possible.

I plan to evolve the back-end to make it easier to write and
maintain for developers writing tests going forward.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 07/19] selftests/firmware: add install target to enable installing test

[Shuah Khan]

On 11/11/2014 06:06 PM, Shuah Khan wrote:
> On 11/11/2014 02:29 PM, Kees Cook wrote:
>> Hi,
>>
>> Sorry, I still really don't like this approach. While it is all in one
>> place (thank you for that), I think it isn't a form that is very
>> workable for the people maintaining the self tests. How about this,
>> instead of per-Makefile customization, why not define an execution
>> framework for these tests instead.
> 
> If I understand correctly, sounds like you don't like the way
> install target is implemented in the individual test Makefiles
> and the changes I made to run_tests targets to address the code
> duplication concern.
> 
> At the moment there is no duplicate code in this patch series
> between install and run_tests targets. This is a  first step
> towards standardizing the framework and a definite improvement
> over what we have currently. As I mentioned earlier, my goal
> is to make it easier for developers to install and run the
> existing tests and evolve the framework as we go.
> 
> Assuming my understanding is correct that:
> 
> -- install and run_tests targets in individual tests can be
>    refined and automated with a common Makefile approach you
>    proposed.
> -- the rest of the user-interface kselftest_install and kselftest
>    are good.
> 
> I would like to propose that we get started with the current
> implementation and refine it based on the following ideas you
> suggested. The refinements you are recommending are confined
> to selftests and can be made after the kselftest_install
> gets added. Adding kselftest_install makes it easier to make
> the refinements as it defines overall UI.
> 

Hi Kees,

Are you ok with the above proposal? I understand this approach
might not be perfect, however it is a step in the right direction
to enable and make it easier to run them. I would like to get some
initial work in for 3.19 if at all possible.

I plan to evolve the back-end to make it easier to write and
maintain for developers writing tests going forward.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org | (970) 217-8978

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Masami Hiramatsu]

(2014/11/12 5:27), Shuah Khan wrote:
> Add a new make target to install to install kernel selftests.
> This new target will build and install selftests. kselftest
> target now depends on kselftest_install and runs the generated
> kselftest script to reduce duplicate work and for common look
> and feel when running tests.
> 
> Approach:
> 
> make kselftest_target:

kselftest_install?

> -- exports kselftest INSTALL_KSFT_PATH
>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> -- exports path for ksefltest.sh
> -- runs selftests make install target:

This direction is OK to me.

BTW, I've found another path to make selftest in Makefile,
Actually you can do

make -C tools/ selftest

And there are selftest_install and selftest_clean targets (but
currently it has a bug and doesn't work, anyway)

I think we'd better do subdir make instead of adding these targets.
This means that "make kselftest*" should be an alias of "make -C tools/ selftest*"

Also, I'd like to request passing some options like as O=$(objtree)
so that we can make test kmodules in selftests.

Thank you,


-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH v2 03/19] selftests: add install target to enable installing selftests

[Masami Hiramatsu]

(2014/11/12 5:27), Shuah Khan wrote:
> Add a new make target to enable installing selftests. This
> new target will call install targets for the tests that are
> specified in INSTALL_TARGETS. During install, a script is
> generated to run tests that are installed. This script will
> be installed in the selftest install directory. Individual
> test Makefiles are changed to add to the script. This will
> allow new tests to add install and run test commands to the
> generated kselftest script. run_tests target runs the
> generated kselftest script to run tests when it is initiated
> from from "make kselftest" from top level source directory.
> 
> Approach:
> 
> make kselftest_target:
> -- exports kselftest INSTALL_KSFT_PATH
>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> -- exports path for ksefltest.sh
> -- runs selftests make install target:
> 
> selftests make install target
> -- creates kselftest.sh script in install install dir
> -- runs install targets for all INSTALL_TARGETS
>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>           to not add more content to patch v1 series. This work
>           will happen soon. In this series these two targets are
>           run after running the generated kselftest script, without
>           any regression in the way these tests are run with
>           "make kselftest" prior to this work.)
> -- install target can be run only from top level source dir.
> 
> Individual test make install targets:
> -- install test programs and/or scripts in install dir
> -- append to the ksefltest.sh file to add commands to run test
> -- install target can be run only from top level source dir.
> 
> Adds the following new ways to initiate selftests:
> -- Installing and running kselftest from install directory
>    by running  "make kselftest"
> -- Running kselftest script from install directory
> 
> Maintains the following ways to run tests:
> -- make -C tools/testing/selftests run_tests
> -- make -C tools/testing/selftests TARGETS=target run_tests
>    Ability specify targets: e.g TARGETS=net
> -- make run_tests from tools/testing/selftests
> -- make run_tests from individual test directories:
>    e.g: make run_tests in tools/testing/selftests/breakpoints
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
>  tools/testing/selftests/Makefile | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
> index 45f145c..b9bdc1d 100644
> --- a/tools/testing/selftests/Makefile
> +++ b/tools/testing/selftests/Makefile
> @@ -16,6 +16,10 @@ TARGETS += sysctl
>  TARGETS += firmware
>  TARGETS += ftrace
>  
> +INSTALL_TARGETS = breakpoints cpu-hotplug efivarfs firmware ipc
> +INSTALL_TARGETS += kcmp memfd memory-hotplug mqueue mount net
> +INSTALL_TARGETS += ptrace sysctl timers user vm
> +
>  TARGETS_HOTPLUG = cpu-hotplug
>  TARGETS_HOTPLUG += memory-hotplug
>  

I think KSELFTEST itself should be defined here, since that is not
a parameter.

> @@ -24,10 +28,35 @@ all:
>  		make -C $$TARGET; \
>  	done;
>  
> -run_tests: all
> +install:
> +ifdef INSTALL_KSFT_PATH
> +	make all
> +	@echo #!/bin/sh\n# Kselftest Run Tests .... >> $(KSELFTEST)
> +	@echo # This file is generated during kselftest_install >> $(KSELFTEST)
> +	@echo # Please don't change it !!\n  >> $(KSELFTEST)
> +	@echo echo ============================== >> $(KSELFTEST)
> +	for TARGET in $(INSTALL_TARGETS); do \
> +		echo Installing $$TARGET; \
> +		make -C $$TARGET install; \

Please pass O= option and others here.

> +	done;
> +	chmod +x $(KSELFTEST)
> +else
> +	@echo Run make kselftest_install in top level source directory
> +endif
> +
> +run_tests:
> +ifdef INSTALL_KSFT_PATH
> +	@cd $(INSTALL_KSFT_PATH); ./kselftest.sh; cd -

We'd better use some macro instead of ./kselftest.sh?

Thank you,

> +# TODO: include ftrace and powerpc in install targets
> +	for TARGET in ftrace powerpc; do \
> +		make -C $$TARGET run_tests; \
> +	done;
> +else
> +	make all
>  	for TARGET in $(TARGETS); do \
>  		make -C $$TARGET run_tests; \
>  	done;
> +endif
>  
>  hotplug:
>  	for TARGET in $(TARGETS_HOTPLUG); do \
> 


-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH v2 11/19] selftests/memory-hotplug: add install target to enable installing test

[Masami Hiramatsu]

(2014/11/12 5:27), Shuah Khan wrote:
> Add a new make target to enable installing test. This target
> installs test in the kselftest install location and add to the
> kselftest script to run the test. Install target can be run
> only from top level source dir.
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
>  tools/testing/selftests/memory-hotplug/Makefile    |  17 +-
>  .../selftests/memory-hotplug/mem-on-off-test.sh    | 238 +++++++++++++++++++++
>  .../selftests/memory-hotplug/on-off-test.sh        | 238 ---------------------
>  3 files changed, 253 insertions(+), 240 deletions(-)
>  create mode 100644 tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
>  delete mode 100644 tools/testing/selftests/memory-hotplug/on-off-test.sh
> 
> diff --git a/tools/testing/selftests/memory-hotplug/Makefile b/tools/testing/selftests/memory-hotplug/Makefile
> index d46b8d4..8921631 100644
> --- a/tools/testing/selftests/memory-hotplug/Makefile
> +++ b/tools/testing/selftests/memory-hotplug/Makefile
> @@ -1,9 +1,22 @@
> +TEST_STR=/bin/bash ./mem-on-off-test.sh -r 2 || echo memory-hotplug selftests: [FAIL]
> +
>  all:
>  
> +install:
> +ifdef INSTALL_KSFT_PATH
> +	install ./mem-on-off-test.sh $(INSTALL_KSFT_PATH)/mem-on-off-test.sh
> +	@echo echo Start memory hotplug test .... >> $(KSELFTEST)
> +	@echo "$(TEST_STR)" >> $(KSELFTEST) >> $(KSELFTEST)
> +	@echo echo End memory hotplug test .... >> $(KSELFTEST)
> +	@echo echo ============================== >> $(KSELFTEST)
> +else
> +	@echo Run make kselftest_install in top level source directory
> +endif

I saw this pattern repeated many times in this series.
Can we make it a macro and include it instead of repeating this code?

Thank you,

-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Michal Marek]

On 2014-11-11 21:27, Shuah Khan wrote:
> Add a new make target to install to install kernel selftests.
> This new target will build and install selftests. kselftest
> target now depends on kselftest_install and runs the generated
> kselftest script to reduce duplicate work and for common look
> and feel when running tests.
> 
> Approach:
> 
> make kselftest_target:
> -- exports kselftest INSTALL_KSFT_PATH
>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> -- exports path for ksefltest.sh
> -- runs selftests make install target:
> 
> selftests make install target
> -- creates kselftest.sh script in install install dir
> -- runs install targets for all INSTALL_TARGETS
>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>           to not add more content to patch v1 series. This work
>           will happen soon. In this series these two targets are
>           run after running the generated kselftest script, without
>           any regression in the way these tests are run with
>           "make kselftest" prior to this work.)
> -- install target can be run only from top level source dir.
> 
> Individual test make install targets:
> -- install test programs and/or scripts in install dir
> -- append to the ksefltest.sh file to add commands to run test
> -- install target can be run only from top level source dir.
> 
> Adds the following new ways to initiate selftests:
> -- Installing and running kselftest from install directory
>    by running  "make kselftest"
> -- Running kselftest script from install directory
> 
> Maintains the following ways to run tests:
> -- make -C tools/testing/selftests run_tests
> -- make -C tools/testing/selftests TARGETS=target run_tests
>    Ability specify targets: e.g TARGETS=net
> -- make run_tests from tools/testing/selftests
> -- make run_tests from individual test directories:
>    e.g: make run_tests in tools/testing/selftests/breakpoints
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> ---
>  Makefile | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 05d67af..ccbd2e1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>  
>  # ---------------------------------------------------------------------------
> -# Kernel selftest
> +# Kernel selftest targets
> +
> +PHONY += __kselftest_configure
> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> +export INSTALL_KSFT_PATH
> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
> +export KSELFTEST

Can this be moved to tools/testing/selftests/Makefile? It's only used in
this part of the tree.


>  PHONY += kselftest
> -kselftest:
> +kselftest: kselftest_install
>  	$(Q)$(MAKE) -C tools/testing/selftests run_tests
>  
> +# Kernel selftest install
> +
> +PHONY += kselftest_install
> +kselftest_install: __kselftest_configure
> +	@rm -rf $(INSTALL_KSFT_PATH)
> +	@mkdir -p $(INSTALL_KSFT_PATH)

Please use $(Q) insteaf od hardcoding the @.


> +	$(Q)$(MAKE) -C tools/testing/selftests install

The install target is only added by the next patch, which in turn
depends on changes done by later patches. The order (in the current
numbering) should be 01/19, 04/19, ... 19/19, 03/19 and 02/19.

Michal

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Michal Marek]

On 2014-11-11 21:27, Shuah Khan wrote:
> Add a new make target to install to install kernel selftests.
> This new target will build and install selftests. kselftest
> target now depends on kselftest_install and runs the generated
> kselftest script to reduce duplicate work and for common look
> and feel when running tests.
> 
> Approach:
> 
> make kselftest_target:
> -- exports kselftest INSTALL_KSFT_PATH
>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> -- exports path for ksefltest.sh
> -- runs selftests make install target:
> 
> selftests make install target
> -- creates kselftest.sh script in install install dir
> -- runs install targets for all INSTALL_TARGETS
>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>           to not add more content to patch v1 series. This work
>           will happen soon. In this series these two targets are
>           run after running the generated kselftest script, without
>           any regression in the way these tests are run with
>           "make kselftest" prior to this work.)
> -- install target can be run only from top level source dir.
> 
> Individual test make install targets:
> -- install test programs and/or scripts in install dir
> -- append to the ksefltest.sh file to add commands to run test
> -- install target can be run only from top level source dir.
> 
> Adds the following new ways to initiate selftests:
> -- Installing and running kselftest from install directory
>    by running  "make kselftest"
> -- Running kselftest script from install directory
> 
> Maintains the following ways to run tests:
> -- make -C tools/testing/selftests run_tests
> -- make -C tools/testing/selftests TARGETS=target run_tests
>    Ability specify targets: e.g TARGETS=net
> -- make run_tests from tools/testing/selftests
> -- make run_tests from individual test directories:
>    e.g: make run_tests in tools/testing/selftests/breakpoints
> 
> Signed-off-by: Shuah Khan <shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org>
> ---
>  Makefile | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 05d67af..ccbd2e1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>  
>  # ---------------------------------------------------------------------------
> -# Kernel selftest
> +# Kernel selftest targets
> +
> +PHONY += __kselftest_configure
> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
> +export INSTALL_KSFT_PATH
> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
> +export KSELFTEST

Can this be moved to tools/testing/selftests/Makefile? It's only used in
this part of the tree.


>  PHONY += kselftest
> -kselftest:
> +kselftest: kselftest_install
>  	$(Q)$(MAKE) -C tools/testing/selftests run_tests
>  
> +# Kernel selftest install
> +
> +PHONY += kselftest_install
> +kselftest_install: __kselftest_configure
> +	@rm -rf $(INSTALL_KSFT_PATH)
> +	@mkdir -p $(INSTALL_KSFT_PATH)

Please use $(Q) insteaf od hardcoding the @.


> +	$(Q)$(MAKE) -C tools/testing/selftests install

The install target is only added by the next patch, which in turn
depends on changes done by later patches. The order (in the current
numbering) should be 01/19, 04/19, ... 19/19, 03/19 and 02/19.

Michal

Re: [PATCH v2 11/19] selftests/memory-hotplug: add install target to enable installing test

[Shuah Khan]

On 11/26/2014 10:49 PM, Masami Hiramatsu wrote:
> (2014/11/12 5:27), Shuah Khan wrote:
>> Add a new make target to enable installing test. This target
>> installs test in the kselftest install location and add to the
>> kselftest script to run the test. Install target can be run
>> only from top level source dir.
>>
>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>> ---
>>  tools/testing/selftests/memory-hotplug/Makefile    |  17 +-
>>  .../selftests/memory-hotplug/mem-on-off-test.sh    | 238 +++++++++++++++++++++
>>  .../selftests/memory-hotplug/on-off-test.sh        | 238 ---------------------
>>  3 files changed, 253 insertions(+), 240 deletions(-)
>>  create mode 100644 tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
>>  delete mode 100644 tools/testing/selftests/memory-hotplug/on-off-test.sh
>>
>> diff --git a/tools/testing/selftests/memory-hotplug/Makefile b/tools/testing/selftests/memory-hotplug/Makefile
>> index d46b8d4..8921631 100644
>> --- a/tools/testing/selftests/memory-hotplug/Makefile
>> +++ b/tools/testing/selftests/memory-hotplug/Makefile
>> @@ -1,9 +1,22 @@
>> +TEST_STR=/bin/bash ./mem-on-off-test.sh -r 2 || echo memory-hotplug selftests: [FAIL]
>> +
>>  all:
>>  
>> +install:
>> +ifdef INSTALL_KSFT_PATH
>> +	install ./mem-on-off-test.sh $(INSTALL_KSFT_PATH)/mem-on-off-test.sh
>> +	@echo echo Start memory hotplug test .... >> $(KSELFTEST)
>> +	@echo "$(TEST_STR)" >> $(KSELFTEST) >> $(KSELFTEST)
>> +	@echo echo End memory hotplug test .... >> $(KSELFTEST)
>> +	@echo echo ============================== >> $(KSELFTEST)
>> +else
>> +	@echo Run make kselftest_install in top level source directory
>> +endif
> 
> I saw this pattern repeated many times in this series.
> Can we make it a macro and include it instead of repeating this code?
> 

I might be able to move this to the selftest Makfile and use target as the
string. I will play with it and see if I can reduce the duplication further.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 03/19] selftests: add install target to enable installing selftests

[Shuah Khan]

On 11/26/2014 10:45 PM, Masami Hiramatsu wrote:
> (2014/11/12 5:27), Shuah Khan wrote:
>> Add a new make target to enable installing selftests. This
>> new target will call install targets for the tests that are
>> specified in INSTALL_TARGETS. During install, a script is
>> generated to run tests that are installed. This script will
>> be installed in the selftest install directory. Individual
>> test Makefiles are changed to add to the script. This will
>> allow new tests to add install and run test commands to the
>> generated kselftest script. run_tests target runs the
>> generated kselftest script to run tests when it is initiated
>> from from "make kselftest" from top level source directory.
>>
>> Approach:
>>
>> make kselftest_target:
>> -- exports kselftest INSTALL_KSFT_PATH
>>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>> -- exports path for ksefltest.sh
>> -- runs selftests make install target:
>>
>> selftests make install target
>> -- creates kselftest.sh script in install install dir
>> -- runs install targets for all INSTALL_TARGETS
>>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>>           to not add more content to patch v1 series. This work
>>           will happen soon. In this series these two targets are
>>           run after running the generated kselftest script, without
>>           any regression in the way these tests are run with
>>           "make kselftest" prior to this work.)
>> -- install target can be run only from top level source dir.
>>
>> Individual test make install targets:
>> -- install test programs and/or scripts in install dir
>> -- append to the ksefltest.sh file to add commands to run test
>> -- install target can be run only from top level source dir.
>>
>> Adds the following new ways to initiate selftests:
>> -- Installing and running kselftest from install directory
>>    by running  "make kselftest"
>> -- Running kselftest script from install directory
>>
>> Maintains the following ways to run tests:
>> -- make -C tools/testing/selftests run_tests
>> -- make -C tools/testing/selftests TARGETS=target run_tests
>>    Ability specify targets: e.g TARGETS=net
>> -- make run_tests from tools/testing/selftests
>> -- make run_tests from individual test directories:
>>    e.g: make run_tests in tools/testing/selftests/breakpoints
>>
>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>> ---
>>  tools/testing/selftests/Makefile | 31 ++++++++++++++++++++++++++++++-
>>  1 file changed, 30 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
>> index 45f145c..b9bdc1d 100644
>> --- a/tools/testing/selftests/Makefile
>> +++ b/tools/testing/selftests/Makefile
>> @@ -16,6 +16,10 @@ TARGETS += sysctl
>>  TARGETS += firmware
>>  TARGETS += ftrace
>>  
>> +INSTALL_TARGETS = breakpoints cpu-hotplug efivarfs firmware ipc
>> +INSTALL_TARGETS += kcmp memfd memory-hotplug mqueue mount net
>> +INSTALL_TARGETS += ptrace sysctl timers user vm
>> +
>>  TARGETS_HOTPLUG = cpu-hotplug
>>  TARGETS_HOTPLUG += memory-hotplug
>>  
> 
> I think KSELFTEST itself should be defined here, since that is not
> a parameter.

I can do that.

> 
>> @@ -24,10 +28,35 @@ all:
>>  		make -C $$TARGET; \
>>  	done;
>>  
>> -run_tests: all
>> +install:
>> +ifdef INSTALL_KSFT_PATH
>> +	make all
>> +	@echo #!/bin/sh\n# Kselftest Run Tests .... >> $(KSELFTEST)
>> +	@echo # This file is generated during kselftest_install >> $(KSELFTEST)
>> +	@echo # Please don't change it !!\n  >> $(KSELFTEST)
>> +	@echo echo ============================== >> $(KSELFTEST)
>> +	for TARGET in $(INSTALL_TARGETS); do \
>> +		echo Installing $$TARGET; \
>> +		make -C $$TARGET install; \
> 
> Please pass O= option and others here.

I will change that.

> 
>> +	done;
>> +	chmod +x $(KSELFTEST)
>> +else
>> +	@echo Run make kselftest_install in top level source directory
>> +endif
>> +
>> +run_tests:
>> +ifdef INSTALL_KSFT_PATH
>> +	@cd $(INSTALL_KSFT_PATH); ./kselftest.sh; cd -
> 
> We'd better use some macro instead of ./kselftest.sh?
> 

I can play with this and see if there is a better way.

thanks,
-- Shuah

-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

On 11/26/2014 10:32 PM, Masami Hiramatsu wrote:
> (2014/11/12 5:27), Shuah Khan wrote:
>> Add a new make target to install to install kernel selftests.
>> This new target will build and install selftests. kselftest
>> target now depends on kselftest_install and runs the generated
>> kselftest script to reduce duplicate work and for common look
>> and feel when running tests.
>>
>> Approach:
>>
>> make kselftest_target:
> 
> kselftest_install?

Thanks for catching this. Will fix it.

> 
>> -- exports kselftest INSTALL_KSFT_PATH
>>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>> -- exports path for ksefltest.sh
>> -- runs selftests make install target:
> 
> This direction is OK to me.
> 
> BTW, I've found another path to make selftest in Makefile,
> Actually you can do
> 
> make -C tools/ selftest
> 
> And there are selftest_install and selftest_clean targets (but
> currently it has a bug and doesn't work, anyway)

Thanks for pointing this out. I didn't know this existed. kind of
hidden.

> 
> I think we'd better do subdir make instead of adding these targets.
> This means that "make kselftest*" should be an alias of "make -C tools/ selftest*"
> 

Yes that would be a good way to go and helps leverage these options
at tools/Makefile level.

> Also, I'd like to request passing some options like as O=$(objtree)
> so that we can make test kmodules in selftests.

Sounds good. I will look into it.

thanks,
-- Shuah

-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

On 12/01/2014 08:47 AM, Michal Marek wrote:
> On 2014-11-11 21:27, Shuah Khan wrote:
>> Add a new make target to install to install kernel selftests.
>> This new target will build and install selftests. kselftest
>> target now depends on kselftest_install and runs the generated
>> kselftest script to reduce duplicate work and for common look
>> and feel when running tests.
>>
>> Approach:
>>
>> make kselftest_target:
>> -- exports kselftest INSTALL_KSFT_PATH
>>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>> -- exports path for ksefltest.sh
>> -- runs selftests make install target:
>>
>> selftests make install target
>> -- creates kselftest.sh script in install install dir
>> -- runs install targets for all INSTALL_TARGETS
>>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>>           to not add more content to patch v1 series. This work
>>           will happen soon. In this series these two targets are
>>           run after running the generated kselftest script, without
>>           any regression in the way these tests are run with
>>           "make kselftest" prior to this work.)
>> -- install target can be run only from top level source dir.
>>
>> Individual test make install targets:
>> -- install test programs and/or scripts in install dir
>> -- append to the ksefltest.sh file to add commands to run test
>> -- install target can be run only from top level source dir.
>>
>> Adds the following new ways to initiate selftests:
>> -- Installing and running kselftest from install directory
>>    by running  "make kselftest"
>> -- Running kselftest script from install directory
>>
>> Maintains the following ways to run tests:
>> -- make -C tools/testing/selftests run_tests
>> -- make -C tools/testing/selftests TARGETS=target run_tests
>>    Ability specify targets: e.g TARGETS=net
>> -- make run_tests from tools/testing/selftests
>> -- make run_tests from individual test directories:
>>    e.g: make run_tests in tools/testing/selftests/breakpoints
>>
>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>> ---
>>  Makefile | 21 +++++++++++++++++++--
>>  1 file changed, 19 insertions(+), 2 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 05d67af..ccbd2e1 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>>  
>>  # ---------------------------------------------------------------------------
>> -# Kernel selftest
>> +# Kernel selftest targets
>> +
>> +PHONY += __kselftest_configure
>> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>> +export INSTALL_KSFT_PATH
>> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
>> +export KSELFTEST
> 
> Can this be moved to tools/testing/selftests/Makefile? It's only used in
> this part of the tree.

I looked into doing that. KERNELRELEASE will have to be exported for
tools/testing/selftests/Makefile to use it? Does that sound okay?

> 
> 
>>  PHONY += kselftest
>> -kselftest:
>> +kselftest: kselftest_install
>>  	$(Q)$(MAKE) -C tools/testing/selftests run_tests
>>  
>> +# Kernel selftest install
>> +
>> +PHONY += kselftest_install
>> +kselftest_install: __kselftest_configure
>> +	@rm -rf $(INSTALL_KSFT_PATH)
>> +	@mkdir -p $(INSTALL_KSFT_PATH)
> 
> Please use $(Q) insteaf od hardcoding the @.

I can do that.

> 
> 
>> +	$(Q)$(MAKE) -C tools/testing/selftests install
> 
> The install target is only added by the next patch, which in turn
> depends on changes done by later patches. The order (in the current
> numbering) should be 01/19, 04/19, ... 19/19, 03/19 and 02/19.
> 

Right. I can re-order the patches in my next version and make other
changes you recommended.

Also, it might be easier to get this series in, if you can Ack the main
Makefile patch (when we are ready i.e), so I can take it through
kselftest tree.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

On 12/01/2014 08:47 AM, Michal Marek wrote:
> On 2014-11-11 21:27, Shuah Khan wrote:
>> Add a new make target to install to install kernel selftests.
>> This new target will build and install selftests. kselftest
>> target now depends on kselftest_install and runs the generated
>> kselftest script to reduce duplicate work and for common look
>> and feel when running tests.
>>
>> Approach:
>>
>> make kselftest_target:
>> -- exports kselftest INSTALL_KSFT_PATH
>>    default $(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>> -- exports path for ksefltest.sh
>> -- runs selftests make install target:
>>
>> selftests make install target
>> -- creates kselftest.sh script in install install dir
>> -- runs install targets for all INSTALL_TARGETS
>>    (Note: ftrace and powerpc aren't included in INSTALL_TARGETS,
>>           to not add more content to patch v1 series. This work
>>           will happen soon. In this series these two targets are
>>           run after running the generated kselftest script, without
>>           any regression in the way these tests are run with
>>           "make kselftest" prior to this work.)
>> -- install target can be run only from top level source dir.
>>
>> Individual test make install targets:
>> -- install test programs and/or scripts in install dir
>> -- append to the ksefltest.sh file to add commands to run test
>> -- install target can be run only from top level source dir.
>>
>> Adds the following new ways to initiate selftests:
>> -- Installing and running kselftest from install directory
>>    by running  "make kselftest"
>> -- Running kselftest script from install directory
>>
>> Maintains the following ways to run tests:
>> -- make -C tools/testing/selftests run_tests
>> -- make -C tools/testing/selftests TARGETS=target run_tests
>>    Ability specify targets: e.g TARGETS=net
>> -- make run_tests from tools/testing/selftests
>> -- make run_tests from individual test directories:
>>    e.g: make run_tests in tools/testing/selftests/breakpoints
>>
>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>> ---
>>  Makefile | 21 +++++++++++++++++++--
>>  1 file changed, 19 insertions(+), 2 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 05d67af..ccbd2e1 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>>  
>>  # ---------------------------------------------------------------------------
>> -# Kernel selftest
>> +# Kernel selftest targets
>> +
>> +PHONY += __kselftest_configure
>> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>> +export INSTALL_KSFT_PATH
>> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
>> +export KSELFTEST
> 
> Can this be moved to tools/testing/selftests/Makefile? It's only used in
> this part of the tree.

I looked into doing that. KERNELRELEASE will have to be exported for
tools/testing/selftests/Makefile to use it? Does that sound okay?

> 
> 
>>  PHONY += kselftest
>> -kselftest:
>> +kselftest: kselftest_install
>>  	$(Q)$(MAKE) -C tools/testing/selftests run_tests
>>  
>> +# Kernel selftest install
>> +
>> +PHONY += kselftest_install
>> +kselftest_install: __kselftest_configure
>> +	@rm -rf $(INSTALL_KSFT_PATH)
>> +	@mkdir -p $(INSTALL_KSFT_PATH)
> 
> Please use $(Q) insteaf od hardcoding the @.

I can do that.

> 
> 
>> +	$(Q)$(MAKE) -C tools/testing/selftests install
> 
> The install target is only added by the next patch, which in turn
> depends on changes done by later patches. The order (in the current
> numbering) should be 01/19, 04/19, ... 19/19, 03/19 and 02/19.
> 

Right. I can re-order the patches in my next version and make other
changes you recommended.

Also, it might be easier to get this series in, if you can Ack the main
Makefile patch (when we are ready i.e), so I can take it through
kselftest tree.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Michal Marek]

On 2014-12-01 17:39, Shuah Khan wrote:
> On 12/01/2014 08:47 AM, Michal Marek wrote:
>> On 2014-11-11 21:27, Shuah Khan wrote:
>>> diff --git a/Makefile b/Makefile
>>> index 05d67af..ccbd2e1 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>>>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>>>  
>>>  # ---------------------------------------------------------------------------
>>> -# Kernel selftest
>>> +# Kernel selftest targets
>>> +
>>> +PHONY += __kselftest_configure
>>> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>>> +export INSTALL_KSFT_PATH
>>> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
>>> +export KSELFTEST
>>
>> Can this be moved to tools/testing/selftests/Makefile? It's only used in
>> this part of the tree.
> 
> I looked into doing that. KERNELRELEASE will have to be exported for
> tools/testing/selftests/Makefile to use it? Does that sound okay?

In fact, KERNELRELEASE is already exported. So go ahead.


> Also, it might be easier to get this series in, if you can Ack the main
> Makefile patch (when we are ready i.e), so I can take it through
> kselftest tree.

Sure. The Makefile change will only consist of redirecting the
kselftest_install target to tools/testing/selftests, right?

Michal

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Michal Marek]

On 2014-12-01 17:39, Shuah Khan wrote:
> On 12/01/2014 08:47 AM, Michal Marek wrote:
>> On 2014-11-11 21:27, Shuah Khan wrote:
>>> diff --git a/Makefile b/Makefile
>>> index 05d67af..ccbd2e1 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>>>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>>>  
>>>  # ---------------------------------------------------------------------------
>>> -# Kernel selftest
>>> +# Kernel selftest targets
>>> +
>>> +PHONY += __kselftest_configure
>>> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>>> +export INSTALL_KSFT_PATH
>>> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
>>> +export KSELFTEST
>>
>> Can this be moved to tools/testing/selftests/Makefile? It's only used in
>> this part of the tree.
> 
> I looked into doing that. KERNELRELEASE will have to be exported for
> tools/testing/selftests/Makefile to use it? Does that sound okay?

In fact, KERNELRELEASE is already exported. So go ahead.


> Also, it might be easier to get this series in, if you can Ack the main
> Makefile patch (when we are ready i.e), so I can take it through
> kselftest tree.

Sure. The Makefile change will only consist of redirecting the
kselftest_install target to tools/testing/selftests, right?

Michal

Re: [PATCH v2 02/19] kbuild: kselftest_install - add a new make target to install selftests

[Shuah Khan]

On 12/03/2014 05:09 AM, Michal Marek wrote:
> On 2014-12-01 17:39, Shuah Khan wrote:
>> On 12/01/2014 08:47 AM, Michal Marek wrote:
>>> On 2014-11-11 21:27, Shuah Khan wrote:
>>>> diff --git a/Makefile b/Makefile
>>>> index 05d67af..ccbd2e1 100644
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -1071,12 +1071,26 @@ headers_check: headers_install
>>>>  	$(Q)$(MAKE) $(hdr-inst)=arch/$(hdr-arch)/include/uapi/asm $(hdr-dst) HDRCHECK=1
>>>>  
>>>>  # ---------------------------------------------------------------------------
>>>> -# Kernel selftest
>>>> +# Kernel selftest targets
>>>> +
>>>> +PHONY += __kselftest_configure
>>>> +INSTALL_KSFT_PATH=$(INSTALL_MOD_PATH)/lib/kselftest/$(KERNELRELEASE)
>>>> +export INSTALL_KSFT_PATH
>>>> +KSELFTEST=$(INSTALL_KSFT_PATH)/kselftest.sh
>>>> +export KSELFTEST
>>>
>>> Can this be moved to tools/testing/selftests/Makefile? It's only used in
>>> this part of the tree.
>>
>> I looked into doing that. KERNELRELEASE will have to be exported for
>> tools/testing/selftests/Makefile to use it? Does that sound okay?
> 
> In fact, KERNELRELEASE is already exported. So go ahead.

Good.

> 
> 
>> Also, it might be easier to get this series in, if you can Ack the main
>> Makefile patch (when we are ready i.e), so I can take it through
>> kselftest tree.
> 
> Sure. The Makefile change will only consist of redirecting the
> kselftest_install target to tools/testing/selftests, right?
> 

Correct. I plan to keep the selftests specific work under the
selftests related Makefiles.

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978