linux-kselftest.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v3] selftests/livepatch: add test skip handling
@ 2019-07-22 14:05 Joe Lawrence
  2019-07-23 15:53 ` shuah
  2019-07-24  6:27 ` [PATCH v3] " Petr Mladek
  0 siblings, 2 replies; 8+ messages in thread
From: Joe Lawrence @ 2019-07-22 14:05 UTC (permalink / raw)
  To: live-patching, linux-kselftest; +Cc: shuah, Jiri Benc

Add a skip() message function that stops the test, logs an explanation,
and sets the "skip" return code (4).

Before loading a livepatch self-test kernel module, first verify that
we've built and installed it by running a 'modprobe --dry-run'.  This
should catch a few environment issues, including !CONFIG_LIVEPATCH and
!CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
skip() function.

Reported-by: Jiri Benc <jbenc@redhat.com>
Suggested-by: Shuah Khan <shuah@kernel.org>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
---

v3: tweak modprobe error message: check kernel config and run as root,
so output now looks like [shuah] :

  % make run_tests
  TAP version 13
  1..3
  # selftests: livepatch: test-livepatch.sh
  # TEST: basic function patching ... SKIP: unable load module test_klp_livepatch, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
  not ok 1 selftests: livepatch: test-livepatch.sh # SKIP
  # selftests: livepatch: test-callbacks.sh
  # TEST: target module before livepatch ... SKIP: unable load module test_klp_callbacks_mod, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
  not ok 2 selftests: livepatch: test-callbacks.sh # SKIP
  # selftests: livepatch: test-shadow-vars.sh
  # TEST: basic shadow variable API ... SKIP: unable load module test_klp_shadow_vars, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
  not ok 3 selftests: livepatch: test-shadow-vars.sh # SKIP

v2: move assert_mod() call into load_mod() and load_lp_nowait(), before
    they check whether the module is a livepatch or not (a test-failing
    assertion). [mbenes, pmladek]

 .../testing/selftests/livepatch/functions.sh  | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh
index 30195449c63c..8eb21fcc71de 100644
--- a/tools/testing/selftests/livepatch/functions.sh
+++ b/tools/testing/selftests/livepatch/functions.sh
@@ -13,6 +13,14 @@ function log() {
 	echo "$1" > /dev/kmsg
 }
 
+# skip(msg) - testing can't proceed
+#	msg - explanation
+function skip() {
+	log "SKIP: $1"
+	echo "SKIP: $1" >&2
+	exit 4
+}
+
 # die(msg) - game over, man
 #	msg - dying words
 function die() {
@@ -43,6 +51,12 @@ function loop_until() {
 	done
 }
 
+function assert_mod() {
+	local mod="$1"
+
+	modprobe --dry-run "$mod" &>/dev/null
+}
+
 function is_livepatch_mod() {
 	local mod="$1"
 
@@ -75,6 +89,9 @@ function __load_mod() {
 function load_mod() {
 	local mod="$1"; shift
 
+	assert_mod "$mod" ||
+		skip "unable load module ${mod}, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root"
+
 	is_livepatch_mod "$mod" &&
 		die "use load_lp() to load the livepatch module $mod"
 
@@ -88,6 +105,9 @@ function load_mod() {
 function load_lp_nowait() {
 	local mod="$1"; shift
 
+	assert_mod "$mod" ||
+		skip "unable load module ${mod}, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root"
+
 	is_livepatch_mod "$mod" ||
 		die "module $mod is not a livepatch"
 
-- 
2.21.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] selftests/livepatch: add test skip handling
  2019-07-22 14:05 [PATCH v3] selftests/livepatch: add test skip handling Joe Lawrence
@ 2019-07-23 15:53 ` shuah
  2019-07-24 20:05   ` [PATCH v4] " Joe Lawrence
  2019-07-24  6:27 ` [PATCH v3] " Petr Mladek
  1 sibling, 1 reply; 8+ messages in thread
From: shuah @ 2019-07-23 15:53 UTC (permalink / raw)
  To: Joe Lawrence, live-patching, linux-kselftest; +Cc: Jiri Benc, shuah

Hi Joe,

On 7/22/19 8:05 AM, Joe Lawrence wrote:
> Add a skip() message function that stops the test, logs an explanation,
> and sets the "skip" return code (4).
> 
> Before loading a livepatch self-test kernel module, first verify that
> we've built and installed it by running a 'modprobe --dry-run'.  This
> should catch a few environment issues, including !CONFIG_LIVEPATCH and
> !CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
> skip() function.
> 
> Reported-by: Jiri Benc <jbenc@redhat.com>
> Suggested-by: Shuah Khan <shuah@kernel.org>
> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
> ---
> 
> v3: tweak modprobe error message: check kernel config and run as root,
> so output now looks like [shuah] :
> 

Couple of small tweaks to user visible message below.

>    % make run_tests
>    TAP version 13
>    1..3
>    # selftests: livepatch: test-livepatch.sh
>    # TEST: basic function patching ... SKIP: unable load module test_klp_livepatch, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
>    not ok 1 selftests: livepatch: test-livepatch.sh # SKIP

unable to

>    # selftests: livepatch: test-callbacks.sh
>    # TEST: target module before livepatch ... SKIP: unable load module test_klp_callbacks_mod, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root

unable to

>    not ok 2 selftests: livepatch: test-callbacks.sh # SKIP
>    # selftests: livepatch: test-shadow-vars.sh
>    # TEST: basic shadow variable API ... SKIP: unable load module test_klp_shadow_vars, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
> 

unable to

thanks for taking care of this.

-- Shuah

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] selftests/livepatch: add test skip handling
  2019-07-22 14:05 [PATCH v3] selftests/livepatch: add test skip handling Joe Lawrence
  2019-07-23 15:53 ` shuah
@ 2019-07-24  6:27 ` Petr Mladek
  2019-07-24 16:45   ` shuah
  1 sibling, 1 reply; 8+ messages in thread
From: Petr Mladek @ 2019-07-24  6:27 UTC (permalink / raw)
  To: Joe Lawrence; +Cc: linux-kselftest, live-patching, shuah, Jiri Benc

On Mon 2019-07-22 10:05:44, Joe Lawrence wrote:
> Add a skip() message function that stops the test, logs an explanation,
> and sets the "skip" return code (4).
> 
> Before loading a livepatch self-test kernel module, first verify that
> we've built and installed it by running a 'modprobe --dry-run'.  This
> should catch a few environment issues, including !CONFIG_LIVEPATCH and
> !CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
> skip() function.
> 
> Reported-by: Jiri Benc <jbenc@redhat.com>
> Suggested-by: Shuah Khan <shuah@kernel.org>
> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>

Reviewed-by: Petr Mladek <pmladek@suse.com>

Best Regards,
Petr

PS: No need to send v4. The missing "to", suggested by Shuah, could
    get added when pushing the patch into the repo.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] selftests/livepatch: add test skip handling
  2019-07-24  6:27 ` [PATCH v3] " Petr Mladek
@ 2019-07-24 16:45   ` shuah
  2019-07-24 18:45     ` Joe Lawrence
  0 siblings, 1 reply; 8+ messages in thread
From: shuah @ 2019-07-24 16:45 UTC (permalink / raw)
  To: Petr Mladek, Joe Lawrence
  Cc: linux-kselftest, live-patching, Jiri Benc, shuah

On 7/24/19 12:27 AM, Petr Mladek wrote:
> On Mon 2019-07-22 10:05:44, Joe Lawrence wrote:
>> Add a skip() message function that stops the test, logs an explanation,
>> and sets the "skip" return code (4).
>>
>> Before loading a livepatch self-test kernel module, first verify that
>> we've built and installed it by running a 'modprobe --dry-run'.  This
>> should catch a few environment issues, including !CONFIG_LIVEPATCH and
>> !CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
>> skip() function.
>>
>> Reported-by: Jiri Benc <jbenc@redhat.com>
>> Suggested-by: Shuah Khan <shuah@kernel.org>
>> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
> 
> Reviewed-by: Petr Mladek <pmladek@suse.com>
> 
> Best Regards,
> Petr
> 
> PS: No need to send v4. The missing "to", suggested by Shuah, could
>      get added when pushing the patch into the repo.
> 

I can't make changes to the patch. I can adjust the commit log at times
and correct merge conflicts.

I would like to see v4 for this.

thanks,
-- Shuah

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] selftests/livepatch: add test skip handling
  2019-07-24 16:45   ` shuah
@ 2019-07-24 18:45     ` Joe Lawrence
  2019-07-24 19:51       ` shuah
  0 siblings, 1 reply; 8+ messages in thread
From: Joe Lawrence @ 2019-07-24 18:45 UTC (permalink / raw)
  To: shuah, Petr Mladek; +Cc: linux-kselftest, live-patching, Jiri Benc

On 07/24/2019 12:45 PM, shuah wrote:
> On 7/24/19 12:27 AM, Petr Mladek wrote:
>> On Mon 2019-07-22 10:05:44, Joe Lawrence wrote:
>>> Add a skip() message function that stops the test, logs an explanation,
>>> and sets the "skip" return code (4).
>>>
>>> Before loading a livepatch self-test kernel module, first verify that
>>> we've built and installed it by running a 'modprobe --dry-run'.  This
>>> should catch a few environment issues, including !CONFIG_LIVEPATCH and
>>> !CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
>>> skip() function.
>>>
>>> Reported-by: Jiri Benc <jbenc@redhat.com>
>>> Suggested-by: Shuah Khan <shuah@kernel.org>
>>> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
>>
>> Reviewed-by: Petr Mladek <pmladek@suse.com>
>>
>> Best Regards,
>> Petr
>>
>> PS: No need to send v4. The missing "to", suggested by Shuah, could
>>      get added when pushing the patch into the repo.
>>
> 
> I can't make changes to the patch. I can adjust the commit log at times
> and correct merge conflicts.
> 
> I would like to see v4 for this.
> 

We should clarify for which tree this and "[PATCH] selftests/livepatch: 
push and pop dynamic debug config​" would be merged.  I had assumed this 
would go to the livepatching tree, but Shuah, do you prefer that 
selftest-centric patches go through the selftests tree?

I'm okay with either and can send up a v4 if needed.  But also this 
would be good to know for future changesets like this, we can perhaps 
mark the destination branch in advance.

Thanks,

-- Joe


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v3] selftests/livepatch: add test skip handling
  2019-07-24 18:45     ` Joe Lawrence
@ 2019-07-24 19:51       ` shuah
  0 siblings, 0 replies; 8+ messages in thread
From: shuah @ 2019-07-24 19:51 UTC (permalink / raw)
  To: Joe Lawrence, Petr Mladek
  Cc: linux-kselftest, live-patching, Jiri Benc, shuah

On 7/24/19 12:45 PM, Joe Lawrence wrote:
> On 07/24/2019 12:45 PM, shuah wrote:
>> On 7/24/19 12:27 AM, Petr Mladek wrote:
>>> On Mon 2019-07-22 10:05:44, Joe Lawrence wrote:
>>>> Add a skip() message function that stops the test, logs an explanation,
>>>> and sets the "skip" return code (4).
>>>>
>>>> Before loading a livepatch self-test kernel module, first verify that
>>>> we've built and installed it by running a 'modprobe --dry-run'.  This
>>>> should catch a few environment issues, including !CONFIG_LIVEPATCH and
>>>> !CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
>>>> skip() function.
>>>>
>>>> Reported-by: Jiri Benc <jbenc@redhat.com>
>>>> Suggested-by: Shuah Khan <shuah@kernel.org>
>>>> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
>>>
>>> Reviewed-by: Petr Mladek <pmladek@suse.com>
>>>
>>> Best Regards,
>>> Petr
>>>
>>> PS: No need to send v4. The missing "to", suggested by Shuah, could
>>>      get added when pushing the patch into the repo.
>>>
>>
>> I can't make changes to the patch. I can adjust the commit log at times
>> and correct merge conflicts.
>>
>> I would like to see v4 for this.
>>
> 
> We should clarify for which tree this and "[PATCH] selftests/livepatch: 
> push and pop dynamic debug config​" would be merged.  I had assumed this 
> would go to the livepatching tree, but Shuah, do you prefer that 
> selftest-centric patches go through the selftests tree?
> 
> I'm okay with either and can send up a v4 if needed.  But also this 
> would be good to know for future changesets like this, we can perhaps 
> mark the destination branch in advance.
> 

I prefer to take these through kselftest tree unless there is a
dependency on other trees. It is the case for some feature
dependent tests that might have dependency on the feature.

This doesn't appear to have any dependency.

thanks,
-- Shuah


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v4] selftests/livepatch: add test skip handling
  2019-07-23 15:53 ` shuah
@ 2019-07-24 20:05   ` Joe Lawrence
  2019-07-24 20:19     ` shuah
  0 siblings, 1 reply; 8+ messages in thread
From: Joe Lawrence @ 2019-07-24 20:05 UTC (permalink / raw)
  To: live-patching, linux-kselftest; +Cc: shuah, Jiri Benc

Add a skip() message function that stops the test, logs an explanation,
and sets the "skip" return code (4).

Before loading a livepatch self-test kernel module, first verify that
we've built and installed it by running a 'modprobe --dry-run'.  This
should catch a few environment issues, including !CONFIG_LIVEPATCH and
!CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
skip() function.

Reported-by: Jiri Benc <jbenc@redhat.com>
Suggested-by: Shuah Khan <shuah@kernel.org>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
---

v4: s/unable load/unable to load/ in user visible skip message. [shuah]

v3: tweak modprobe error message: check kernel config and run as root,
so output now looks like [shuah] :

  % make run_tests
  TAP version 13
  1..3
  # selftests: livepatch: test-livepatch.sh
  # TEST: basic function patching ... SKIP: unable load module test_klp_livepatch, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
  not ok 1 selftests: livepatch: test-livepatch.sh # SKIP
  # selftests: livepatch: test-callbacks.sh
  # TEST: target module before livepatch ... SKIP: unable load module test_klp_callbacks_mod, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
  not ok 2 selftests: livepatch: test-callbacks.sh # SKIP
  # selftests: livepatch: test-shadow-vars.sh
  # TEST: basic shadow variable API ... SKIP: unable load module test_klp_shadow_vars, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root
  not ok 3 selftests: livepatch: test-shadow-vars.sh # SKIP

v2: move assert_mod() call into load_mod() and load_lp_nowait(), before
    they check whether the module is a livepatch or not (a test-failing
    assertion). [mbenes, pmladek]


 .../testing/selftests/livepatch/functions.sh  | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh
index 30195449c63c..edcfeace4655 100644
--- a/tools/testing/selftests/livepatch/functions.sh
+++ b/tools/testing/selftests/livepatch/functions.sh
@@ -13,6 +13,14 @@ function log() {
 	echo "$1" > /dev/kmsg
 }
 
+# skip(msg) - testing can't proceed
+#	msg - explanation
+function skip() {
+	log "SKIP: $1"
+	echo "SKIP: $1" >&2
+	exit 4
+}
+
 # die(msg) - game over, man
 #	msg - dying words
 function die() {
@@ -43,6 +51,12 @@ function loop_until() {
 	done
 }
 
+function assert_mod() {
+	local mod="$1"
+
+	modprobe --dry-run "$mod" &>/dev/null
+}
+
 function is_livepatch_mod() {
 	local mod="$1"
 
@@ -75,6 +89,9 @@ function __load_mod() {
 function load_mod() {
 	local mod="$1"; shift
 
+	assert_mod "$mod" ||
+		skip "unable to load module ${mod}, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root"
+
 	is_livepatch_mod "$mod" &&
 		die "use load_lp() to load the livepatch module $mod"
 
@@ -88,6 +105,9 @@ function load_mod() {
 function load_lp_nowait() {
 	local mod="$1"; shift
 
+	assert_mod "$mod" ||
+		skip "unable to load module ${mod}, verify CONFIG_TEST_LIVEPATCH=m and run self-tests as root"
+
 	is_livepatch_mod "$mod" ||
 		die "module $mod is not a livepatch"
 
-- 
2.21.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH v4] selftests/livepatch: add test skip handling
  2019-07-24 20:05   ` [PATCH v4] " Joe Lawrence
@ 2019-07-24 20:19     ` shuah
  0 siblings, 0 replies; 8+ messages in thread
From: shuah @ 2019-07-24 20:19 UTC (permalink / raw)
  To: Joe Lawrence, live-patching, linux-kselftest; +Cc: Jiri Benc, shuah

On 7/24/19 2:05 PM, Joe Lawrence wrote:
> Add a skip() message function that stops the test, logs an explanation,
> and sets the "skip" return code (4).
> 
> Before loading a livepatch self-test kernel module, first verify that
> we've built and installed it by running a 'modprobe --dry-run'.  This
> should catch a few environment issues, including !CONFIG_LIVEPATCH and
> !CONFIG_TEST_LIVEPATCH.  In these cases, exit gracefully with the new
> skip() function.
> 
> Reported-by: Jiri Benc <jbenc@redhat.com>
> Suggested-by: Shuah Khan <shuah@kernel.org>
> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
> ---
> 
> v4: s/unable load/unable to load/ in user visible skip message. [shuah]
> 

Thanks. Applied to linux-ksefktest fixes for 5.3-rc2 with Petr's Ack

thanks,
-- Shuah

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2019-07-24 20:19 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-22 14:05 [PATCH v3] selftests/livepatch: add test skip handling Joe Lawrence
2019-07-23 15:53 ` shuah
2019-07-24 20:05   ` [PATCH v4] " Joe Lawrence
2019-07-24 20:19     ` shuah
2019-07-24  6:27 ` [PATCH v3] " Petr Mladek
2019-07-24 16:45   ` shuah
2019-07-24 18:45     ` Joe Lawrence
2019-07-24 19:51       ` shuah

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).