[PATCH v4 0/2] Create specific kconfig for kselftest

[PATCH v4 0/2] Create specific kconfig for kselftest

[Bamvor Jian Zhang]

There is a discussion about improving the usability of kselftest by
creating test-specific kconfig in recent kernel Summit. The first
three versions could be found at [1], [2], [3].

Patch 1 keep test-specific kconfig fragments inside each selftest so
that merge_configs.sh could build up a kernel configuration that can
test the specific or all feature(s).

Patch 2 add config option(kselftest-merge) in make file as a
helper to merge all the test config dependecies to .config.

Changes since v3:
1.  move the target kselftest-mergeconfig from scripts/kconfig/Makfile
    to toplevel Makefile. And rename to kselftest-merge to avoid the
    conflict with "%config" target.

Changes since v2:
1.  Fix the misuse of objtree and srctree in the
    kselftest-mergeconfig target.

Changes since v1:
1.  Add kselftest-mergeconfig in scripts/kconfig/Makefile according
    to the suggestion from Michael.

[1] http://www.spinics.net/lists/linux-api/msg15533.html
[2] http://www.spinics.net/lists/linux-api/msg15593.html
[3] http://comments.gmane.org/gmane.linux.kernel/2110565

Bamvor Jian Zhang (2):
  selftests: create test-specific kconfig fragments
  Makefile: add kselftest-merge

 Makefile                                      | 11 +++++++++++
 tools/testing/selftests/cpu-hotplug/config    |  2 ++
 tools/testing/selftests/firmware/config       |  1 +
 tools/testing/selftests/ftrace/config         |  1 +
 tools/testing/selftests/ipc/config            |  2 ++
 tools/testing/selftests/memory-hotplug/config |  4 ++++
 tools/testing/selftests/mount/config          |  2 ++
 tools/testing/selftests/net/config            |  3 +++
 tools/testing/selftests/pstore/config         |  4 ++++
 tools/testing/selftests/seccomp/config        |  2 ++
 tools/testing/selftests/static_keys/config    |  1 +
 tools/testing/selftests/user/config           |  1 +
 tools/testing/selftests/vm/config             |  1 +
 tools/testing/selftests/zram/config           |  2 ++
 14 files changed, 37 insertions(+)
 create mode 100644 tools/testing/selftests/cpu-hotplug/config
 create mode 100644 tools/testing/selftests/firmware/config
 create mode 100644 tools/testing/selftests/ftrace/config
 create mode 100644 tools/testing/selftests/ipc/config
 create mode 100644 tools/testing/selftests/memory-hotplug/config
 create mode 100644 tools/testing/selftests/mount/config
 create mode 100644 tools/testing/selftests/net/config
 create mode 100644 tools/testing/selftests/pstore/config
 create mode 100644 tools/testing/selftests/seccomp/config
 create mode 100644 tools/testing/selftests/static_keys/config
 create mode 100644 tools/testing/selftests/user/config
 create mode 100644 tools/testing/selftests/vm/config
 create mode 100644 tools/testing/selftests/zram/config

-- 
2.1.4

[PATCH v4 1/2] selftests: create test-specific kconfig fragments

[Bamvor Jian Zhang]

Create the config file in each directory of testcase which need
more kernel configuration than the default defconfig. User could
use these configs with merge_config.sh script:

Enable config for specific testcase:
(export ARCH=xxx #for cross compiling)
./scripts/kconfig/merge_config.sh .config \
		tools/testing/selftests/xxx/config

Enable configs for all testcases:
(export ARCH=xxx #for cross compiling)
./scripts/kconfig/merge_config.sh .config \
		tools/testing/selftests/*/config

Signed-off-by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Reviewed-by: Shuah Khan <shuahkh@osg.samsung.com>
---
 tools/testing/selftests/cpu-hotplug/config    | 2 ++
 tools/testing/selftests/firmware/config       | 1 +
 tools/testing/selftests/ftrace/config         | 1 +
 tools/testing/selftests/ipc/config            | 2 ++
 tools/testing/selftests/memory-hotplug/config | 4 ++++
 tools/testing/selftests/mount/config          | 2 ++
 tools/testing/selftests/net/config            | 3 +++
 tools/testing/selftests/pstore/config         | 4 ++++
 tools/testing/selftests/seccomp/config        | 2 ++
 tools/testing/selftests/static_keys/config    | 1 +
 tools/testing/selftests/user/config           | 1 +
 tools/testing/selftests/vm/config             | 1 +
 tools/testing/selftests/zram/config           | 2 ++
 13 files changed, 26 insertions(+)
 create mode 100644 tools/testing/selftests/cpu-hotplug/config
 create mode 100644 tools/testing/selftests/firmware/config
 create mode 100644 tools/testing/selftests/ftrace/config
 create mode 100644 tools/testing/selftests/ipc/config
 create mode 100644 tools/testing/selftests/memory-hotplug/config
 create mode 100644 tools/testing/selftests/mount/config
 create mode 100644 tools/testing/selftests/net/config
 create mode 100644 tools/testing/selftests/pstore/config
 create mode 100644 tools/testing/selftests/seccomp/config
 create mode 100644 tools/testing/selftests/static_keys/config
 create mode 100644 tools/testing/selftests/user/config
 create mode 100644 tools/testing/selftests/vm/config
 create mode 100644 tools/testing/selftests/zram/config

diff --git a/tools/testing/selftests/cpu-hotplug/config b/tools/testing/selftests/cpu-hotplug/config
new file mode 100644
index 0000000..e6ab090
--- /dev/null
+++ b/tools/testing/selftests/cpu-hotplug/config
@@ -0,0 +1,2 @@
+CONFIG_NOTIFIER_ERROR_INJECTION=y
+CONFIG_CPU_NOTIFIER_ERROR_INJECT=m
diff --git a/tools/testing/selftests/firmware/config b/tools/testing/selftests/firmware/config
new file mode 100644
index 0000000..c8137f7
--- /dev/null
+++ b/tools/testing/selftests/firmware/config
@@ -0,0 +1 @@
+CONFIG_TEST_FIRMWARE=y
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
new file mode 100644
index 0000000..ef82146
--- /dev/null
+++ b/tools/testing/selftests/ftrace/config
@@ -0,0 +1 @@
+CONFIG_FTRACE=y
diff --git a/tools/testing/selftests/ipc/config b/tools/testing/selftests/ipc/config
new file mode 100644
index 0000000..0702447
--- /dev/null
+++ b/tools/testing/selftests/ipc/config
@@ -0,0 +1,2 @@
+CONFIG_EXPERT=y
+CONFIG_CHECKPOINT_RESTORE=y
diff --git a/tools/testing/selftests/memory-hotplug/config b/tools/testing/selftests/memory-hotplug/config
new file mode 100644
index 0000000..2fde301
--- /dev/null
+++ b/tools/testing/selftests/memory-hotplug/config
@@ -0,0 +1,4 @@
+CONFIG_MEMORY_HOTPLUG=y
+CONFIG_MEMORY_HOTPLUG_SPARSE=y
+CONFIG_NOTIFIER_ERROR_INJECTION=y
+CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m
diff --git a/tools/testing/selftests/mount/config b/tools/testing/selftests/mount/config
new file mode 100644
index 0000000..b5d881e
--- /dev/null
+++ b/tools/testing/selftests/mount/config
@@ -0,0 +1,2 @@
+CONFIG_USER_NS=y
+CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
diff --git a/tools/testing/selftests/net/config b/tools/testing/selftests/net/config
new file mode 100644
index 0000000..e57b4ac
--- /dev/null
+++ b/tools/testing/selftests/net/config
@@ -0,0 +1,3 @@
+CONFIG_USER_NS=y
+CONFIG_BPF_SYSCALL=y
+CONFIG_TEST_BPF=m
diff --git a/tools/testing/selftests/pstore/config b/tools/testing/selftests/pstore/config
new file mode 100644
index 0000000..6a8e5a9
--- /dev/null
+++ b/tools/testing/selftests/pstore/config
@@ -0,0 +1,4 @@
+CONFIG_MISC_FILESYSTEMS=y
+CONFIG_PSTORE=y
+CONFIG_PSTORE_PMSG=y
+CONFIG_PSTORE_CONSOLE=y
diff --git a/tools/testing/selftests/seccomp/config b/tools/testing/selftests/seccomp/config
new file mode 100644
index 0000000..db1e11b
--- /dev/null
+++ b/tools/testing/selftests/seccomp/config
@@ -0,0 +1,2 @@
+CONFIG_SECCOMP=y
+CONFIG_SECCOMP_FILTER=y
diff --git a/tools/testing/selftests/static_keys/config b/tools/testing/selftests/static_keys/config
new file mode 100644
index 0000000..d538fb7
--- /dev/null
+++ b/tools/testing/selftests/static_keys/config
@@ -0,0 +1 @@
+CONFIG_TEST_STATIC_KEYS=m
diff --git a/tools/testing/selftests/user/config b/tools/testing/selftests/user/config
new file mode 100644
index 0000000..784ed84
--- /dev/null
+++ b/tools/testing/selftests/user/config
@@ -0,0 +1 @@
+CONFIG_TEST_USER_COPY=m
diff --git a/tools/testing/selftests/vm/config b/tools/testing/selftests/vm/config
new file mode 100644
index 0000000..698c7ed
--- /dev/null
+++ b/tools/testing/selftests/vm/config
@@ -0,0 +1 @@
+CONFIG_USERFAULTFD=y
diff --git a/tools/testing/selftests/zram/config b/tools/testing/selftests/zram/config
new file mode 100644
index 0000000..e0cc47e
--- /dev/null
+++ b/tools/testing/selftests/zram/config
@@ -0,0 +1,2 @@
+CONFIG_ZSMALLOC=y
+CONFIG_ZRAM=m
-- 
2.1.4

[PATCH v4 2/2] Makefile: add kselftest-merge

[Bamvor Jian Zhang]

Add kselftest-merge to enable the dependencies of kernel config
for all the kselftest testcases. User should call it after create
the kernel configuration(.config), eg:

$ make kselftest-merge

Signed-off-by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
---
 Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/Makefile b/Makefile
index cd40956..bfb9faa 100644
--- a/Makefile
+++ b/Makefile
@@ -1104,6 +1104,14 @@ kselftest:
 kselftest-clean:
 	$(Q)$(MAKE) -C tools/testing/selftests clean
 
+PHONY += kselftest-merge
+kselftest-merge:
+	$(if $(wildcard $(objtree)/.config),, $(error No .config exists, config your kernel first!))
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh \
+		-m $(objtree)/.config \
+		$(srctree)/tools/testing/selftests/*/config
+	+$(Q)$(MAKE) -f $(srctree)/Makefile olddefconfig
+
 # ---------------------------------------------------------------------------
 # Modules
 
@@ -1312,6 +1320,8 @@ help:
 	@echo  '                    Build, install, and boot kernel before'
 	@echo  '                    running kselftest on it'
 	@echo  '  kselftest-clean - Remove all generated kselftest files'
+	@echo  '  kselftest-merge - Merge all the config dependencies of kselftest to existed'
+	@echo  '                    .config.'
 	@echo  ''
 	@echo  'Kernel packaging:'
 	@$(MAKE) $(build)=$(package-dir) help
-- 
2.1.4

Re: [PATCH v4 0/2] Create specific kconfig for kselftest

[Shuah Khan]

On 01/08/2016 12:27 AM, Bamvor Jian Zhang wrote:
> There is a discussion about improving the usability of kselftest by
> creating test-specific kconfig in recent kernel Summit. The first
> three versions could be found at [1], [2], [3].
> 
> Patch 1 keep test-specific kconfig fragments inside each selftest so
> that merge_configs.sh could build up a kernel configuration that can
> test the specific or all feature(s).
> 
> Patch 2 add config option(kselftest-merge) in make file as a
> helper to merge all the test config dependecies to .config.

Hi Michal,

Are you okay with the changes to the second patch
in this series? Would you like to me to send the
second one up through linux-kselftest tree or do
you want to take this through your tree?

Please let me know.

thanks,
-- Shuah
> 
> Changes since v3:
> 1.  move the target kselftest-mergeconfig from scripts/kconfig/Makfile
>     to toplevel Makefile. And rename to kselftest-merge to avoid the
>     conflict with "%config" target.
> 
> Changes since v2:
> 1.  Fix the misuse of objtree and srctree in the
>     kselftest-mergeconfig target.
> 
> Changes since v1:
> 1.  Add kselftest-mergeconfig in scripts/kconfig/Makefile according
>     to the suggestion from Michael.
> 
> [1] http://www.spinics.net/lists/linux-api/msg15533.html
> [2] http://www.spinics.net/lists/linux-api/msg15593.html
> [3] http://comments.gmane.org/gmane.linux.kernel/2110565
> 
> Bamvor Jian Zhang (2):
>   selftests: create test-specific kconfig fragments
>   Makefile: add kselftest-merge
> 
>  Makefile                                      | 11 +++++++++++
>  tools/testing/selftests/cpu-hotplug/config    |  2 ++
>  tools/testing/selftests/firmware/config       |  1 +
>  tools/testing/selftests/ftrace/config         |  1 +
>  tools/testing/selftests/ipc/config            |  2 ++
>  tools/testing/selftests/memory-hotplug/config |  4 ++++
>  tools/testing/selftests/mount/config          |  2 ++
>  tools/testing/selftests/net/config            |  3 +++
>  tools/testing/selftests/pstore/config         |  4 ++++
>  tools/testing/selftests/seccomp/config        |  2 ++
>  tools/testing/selftests/static_keys/config    |  1 +
>  tools/testing/selftests/user/config           |  1 +
>  tools/testing/selftests/vm/config             |  1 +
>  tools/testing/selftests/zram/config           |  2 ++
>  14 files changed, 37 insertions(+)
>  create mode 100644 tools/testing/selftests/cpu-hotplug/config
>  create mode 100644 tools/testing/selftests/firmware/config
>  create mode 100644 tools/testing/selftests/ftrace/config
>  create mode 100644 tools/testing/selftests/ipc/config
>  create mode 100644 tools/testing/selftests/memory-hotplug/config
>  create mode 100644 tools/testing/selftests/mount/config
>  create mode 100644 tools/testing/selftests/net/config
>  create mode 100644 tools/testing/selftests/pstore/config
>  create mode 100644 tools/testing/selftests/seccomp/config
>  create mode 100644 tools/testing/selftests/static_keys/config
>  create mode 100644 tools/testing/selftests/user/config
>  create mode 100644 tools/testing/selftests/vm/config
>  create mode 100644 tools/testing/selftests/zram/config
> 


-- 
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v4 2/2] Makefile: add kselftest-merge

[Shuah Khan]

On 01/08/2016 12:27 AM, Bamvor Jian Zhang wrote:
> Add kselftest-merge to enable the dependencies of kernel config
> for all the kselftest testcases. User should call it after create
> the kernel configuration(.config), eg:
> 
> $ make kselftest-merge
> 
> Signed-off-by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>

Could you please ACK this patch if you are okay
with the changes? I will take this through ksleftest
tree.

thanks,
-- Shuah

> ---
>  Makefile | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index cd40956..bfb9faa 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1104,6 +1104,14 @@ kselftest:
>  kselftest-clean:
>  	$(Q)$(MAKE) -C tools/testing/selftests clean
>  
> +PHONY += kselftest-merge
> +kselftest-merge:
> +	$(if $(wildcard $(objtree)/.config),, $(error No .config exists, config your kernel first!))
> +	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh \
> +		-m $(objtree)/.config \
> +		$(srctree)/tools/testing/selftests/*/config
> +	+$(Q)$(MAKE) -f $(srctree)/Makefile olddefconfig
> +
>  # ---------------------------------------------------------------------------
>  # Modules
>  
> @@ -1312,6 +1320,8 @@ help:
>  	@echo  '                    Build, install, and boot kernel before'
>  	@echo  '                    running kselftest on it'
>  	@echo  '  kselftest-clean - Remove all generated kselftest files'
> +	@echo  '  kselftest-merge - Merge all the config dependencies of kselftest to existed'
> +	@echo  '                    .config.'
>  	@echo  ''
>  	@echo  'Kernel packaging:'
>  	@$(MAKE) $(build)=$(package-dir) help
> 


-- 
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v4 1/2] selftests: create test-specific kconfig fragments

[Shuah Khan]

On 01/08/2016 12:27 AM, Bamvor Jian Zhang wrote:
> Create the config file in each directory of testcase which need
> more kernel configuration than the default defconfig. User could
> use these configs with merge_config.sh script:
> 
> Enable config for specific testcase:
> (export ARCH=xxx #for cross compiling)
> ./scripts/kconfig/merge_config.sh .config \
> 		tools/testing/selftests/xxx/config
> 
> Enable configs for all testcases:
> (export ARCH=xxx #for cross compiling)
> ./scripts/kconfig/merge_config.sh .config \
> 		tools/testing/selftests/*/config
> 
> Signed-off-by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
> Reviewed-by: Shuah Khan <shuahkh@osg.samsung.com>

Applied to linux-kselftest next for 4.6-rc1

thanks,
-- Shuah


-- 
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978

Re: [PATCH v4 2/2] Makefile: add kselftest-merge

[Shuah Khan]

On 02/23/2016 07:34 AM, Shuah Khan wrote:
> On 01/08/2016 12:27 AM, Bamvor Jian Zhang wrote:
>> Add kselftest-merge to enable the dependencies of kernel config
>> for all the kselftest testcases. User should call it after create
>> the kernel configuration(.config), eg:
>>
>> $ make kselftest-merge
>>
>> Signed-off-by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
> 
> Could you please ACK this patch if you are okay
> with the changes? I will take this through ksleftest
> tree.
> 
> thanks,
> -- Shuah

Applied this patch to linux-kselftest next with

[PATCH v4 1/2] selftests: create test-specific kconfig fragments

to get some testing in for now.

thanks,
-- Shuah

-- 
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978