[RFC PATCH] selftests/livepatch: only consider supported arches

[RFC PATCH] selftests/livepatch: only consider supported arches

[Joe Lawrence]

Only run the livepatching self-tests on x86_64 and ppc64le arches.

Reported-by: Jiri Benc <jbenc@redhat.com>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
---
 tools/testing/selftests/livepatch/Makefile | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/tools/testing/selftests/livepatch/Makefile b/tools/testing/selftests/livepatch/Makefile
index fd405402c3ff..9842fb473d42 100644
--- a/tools/testing/selftests/livepatch/Makefile
+++ b/tools/testing/selftests/livepatch/Makefile
@@ -1,5 +1,16 @@
 # SPDX-License-Identifier: GPL-2.0
 
+ARCH ?= $(shell uname -m 2>/dev/null || echo not)
+
+ifeq ($(ARCH),x86_64)
+livepatch_supported := 1
+endif
+ifeq ($(ARCH),ppc64le)
+livepatch_supported := 1
+endif
+
+ifdef livepatch_supported
+
 TEST_PROGS_EXTENDED := functions.sh
 TEST_PROGS := \
 	test-livepatch.sh \
@@ -7,3 +18,5 @@ TEST_PROGS := \
 	test-shadow-vars.sh
 
 include ../lib.mk
+
+endif
-- 
2.21.0

Re: [RFC PATCH] selftests/livepatch: only consider supported arches

[Joe Lawrence]

On Fri, Jul 12, 2019 at 01:14:02PM -0400, Joe Lawrence wrote:
> Only run the livepatching self-tests on x86_64 and ppc64le arches.
> 
> Reported-by: Jiri Benc <jbenc@redhat.com>
> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
> ---
>  tools/testing/selftests/livepatch/Makefile | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/tools/testing/selftests/livepatch/Makefile b/tools/testing/selftests/livepatch/Makefile
> index fd405402c3ff..9842fb473d42 100644
> --- a/tools/testing/selftests/livepatch/Makefile
> +++ b/tools/testing/selftests/livepatch/Makefile
> @@ -1,5 +1,16 @@
>  # SPDX-License-Identifier: GPL-2.0
>  
> +ARCH ?= $(shell uname -m 2>/dev/null || echo not)
> +
> +ifeq ($(ARCH),x86_64)
> +livepatch_supported := 1
> +endif
> +ifeq ($(ARCH),ppc64le)
> +livepatch_supported := 1
> +endif
> +
> +ifdef livepatch_supported
> +
>  TEST_PROGS_EXTENDED := functions.sh
>  TEST_PROGS := \
>  	test-livepatch.sh \
> @@ -7,3 +18,5 @@ TEST_PROGS := \
>  	test-shadow-vars.sh
>  
>  include ../lib.mk
> +
> +endif
> -- 
> 2.21.0
> 

Hi Shuah,

This was change was a quick and dirty fix to avoid inadvertent errors
reported by run_kselftest.sh on arches unsupported by livepatch.

Ideally we would check CONFIG_TEST_LIVEPATCH instead of the arches
directly:  This would allow distros to "turn off" this particular set of
selftests through the kernel configuration.  Also, setting 
CONFIG_TEST_LIVEPATCH=m determines whether supporting livepatch
self-test modules (in lib/livepatch) will be built.  Without it, these
self-test scripts are useless.

However, I didn't see that $(CONFIG_TEST_LIVEPATCH) was set when
invoking tools/testing/selftests/livepatch/Makefile.  I don't know how
tools/testing/selftests/vDSO/Makefile is using $(CONFIG_X86_32) but
perhaps that isn't working either.

Any suggestions?  Should we modify the test scripts themselves to deal
with a !CONFIG_TEST_LIVEPATCH environment?

Thanks,

-- Joe

Re: [RFC PATCH] selftests/livepatch: only consider supported arches

[shuah]

Hi Joe,

On 7/12/19 11:25 AM, Joe Lawrence wrote:
> On Fri, Jul 12, 2019 at 01:14:02PM -0400, Joe Lawrence wrote:
>> Only run the livepatching self-tests on x86_64 and ppc64le arches.
>>
>> Reported-by: Jiri Benc <jbenc@redhat.com>
>> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
>> ---
>>   tools/testing/selftests/livepatch/Makefile | 13 +++++++++++++
>>   1 file changed, 13 insertions(+)
>>
>> diff --git a/tools/testing/selftests/livepatch/Makefile b/tools/testing/selftests/livepatch/Makefile
>> index fd405402c3ff..9842fb473d42 100644
>> --- a/tools/testing/selftests/livepatch/Makefile
>> +++ b/tools/testing/selftests/livepatch/Makefile
>> @@ -1,5 +1,16 @@
>>   # SPDX-License-Identifier: GPL-2.0
>>   
>> +ARCH ?= $(shell uname -m 2>/dev/null || echo not)
>> +
>> +ifeq ($(ARCH),x86_64)
>> +livepatch_supported := 1
>> +endif
>> +ifeq ($(ARCH),ppc64le)
>> +livepatch_supported := 1
>> +endif
>> +
>> +ifdef livepatch_supported
>> +
>>   TEST_PROGS_EXTENDED := functions.sh
>>   TEST_PROGS := \
>>   	test-livepatch.sh \
>> @@ -7,3 +18,5 @@ TEST_PROGS := \
>>   	test-shadow-vars.sh
>>   
>>   include ../lib.mk
>> +
>> +endif
>> -- 
>> 2.21.0
>>
> 
> Hi Shuah,
> 
> This was change was a quick and dirty fix to avoid inadvertent errors
> reported by run_kselftest.sh on arches unsupported by livepatch.
> 
> Ideally we would check CONFIG_TEST_LIVEPATCH instead of the arches
> directly:  This would allow distros to "turn off" this particular set of
> selftests through the kernel configuration.  Also, setting
> CONFIG_TEST_LIVEPATCH=m determines whether supporting livepatch
> self-test modules (in lib/livepatch) will be built.  Without it, these
> self-test scripts are useless.
> 

I was just about to send you email about looking into using config
fragments.

Looks like there is a tools/testing/selftests/livepatch/config

CONFIG_TEST_LIVEPATCH=m

> However, I didn't see that $(CONFIG_TEST_LIVEPATCH) was set when
> invoking tools/testing/selftests/livepatch/Makefile.  I don't know how
> tools/testing/selftests/vDSO/Makefile is using $(CONFIG_X86_32) but
> perhaps that isn't working either.
> 
> Any suggestions?  Should we modify the test scripts themselves to deal
> with a !CONFIG_TEST_LIVEPATCH environment?
> 

Looks livepatch tests are shell scripts and there is nothing to build.
So the build time check of having a config fragment won't work for this
case.

test-livepatch.sh loads test_klp_livepatch and test_klp_atomic_replace
functions.sh has the handling for modprobing livepatch module.

The one thing I am not seeing is Skip handling. Without that users will
complain livepatch test is failing as opposed to that the test can't
run due to unmet dependencies and skipped.

Maybe that is all you need? I would recommend going in that direction
instead of Arch check.

thanks,
-- Shuah

Re: [RFC PATCH] selftests/livepatch: only consider supported arches

[Joe Lawrence]

On Fri, Jul 12, 2019 at 11:43:02AM -0600, shuah wrote:
> Hi Joe,
> 
> [ ... snip ... ]
> 
> The one thing I am not seeing is Skip handling. Without that users will
> complain livepatch test is failing as opposed to that the test can't
> run due to unmet dependencies and skipped.
> 
> Maybe that is all you need? I would recommend going in that direction
> instead of Arch check.
> 

Okay, I see that kselftest/runner.sh compares test return code with
skip_rc=4 to determine SKIP status... so perhaps our scripts could
perform a simple "modinfo" on their test modules to determine if they
have been built and installed?  If not found, just SKIP to the next
test.

Thanks, 

-- Joe

Re: [RFC PATCH] selftests/livepatch: only consider supported arches

[shuah]

On 7/12/19 11:58 AM, Joe Lawrence wrote:
> On Fri, Jul 12, 2019 at 11:43:02AM -0600, shuah wrote:
>> Hi Joe,
>>
>> [ ... snip ... ]
>>
>> The one thing I am not seeing is Skip handling. Without that users will
>> complain livepatch test is failing as opposed to that the test can't
>> run due to unmet dependencies and skipped.
>>
>> Maybe that is all you need? I would recommend going in that direction
>> instead of Arch check.
>>
> 
> Okay, I see that kselftest/runner.sh compares test return code with
> skip_rc=4 to determine SKIP status... so perhaps our scripts could
> perform a simple "modinfo" on their test modules to determine if they
> have been built and installed?  If not found, just SKIP to the next
> test.
> 

Yes. That would fix the problem.

tools/testing/selftests/kselftest_module.sh has hooks to help you
with modules checks. e.g: assert_have_module

thanks,
-- Shuah