live-patching.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Petr Mladek <pmladek@suse.com>
To: Joe Lawrence <joe.lawrence@redhat.com>
Cc: live-patching@vger.kernel.org, linux-kselftest@vger.kernel.org
Subject: Re: [PATCH 1/3] selftests/livepatch: Don't clear dmesg when running tests
Date: Fri, 12 Jun 2020 11:49:03 +0200	[thread overview]
Message-ID: <20200612094903.GE4311@linux-b0ei> (raw)
In-Reply-To: <20200610172101.21910-2-joe.lawrence@redhat.com>

On Wed 2020-06-10 13:20:59, Joe Lawrence wrote:
> Inspired by commit f131d9edc29d ("selftests/lkdtm: Don't clear dmesg
> when running tests"), keep a reference dmesg copy when beginning each
> test.  This way check_result() can compare against the initial copy
> rather than relying upon an empty log.
> 
> Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
> ---
>  tools/testing/selftests/livepatch/README      | 16 +++---
>  .../testing/selftests/livepatch/functions.sh  | 16 +++++-
>  .../selftests/livepatch/test-callbacks.sh     | 55 ++++---------------
>  .../selftests/livepatch/test-ftrace.sh        |  5 +-
>  .../selftests/livepatch/test-livepatch.sh     | 15 +----
>  .../selftests/livepatch/test-shadow-vars.sh   |  5 +-
>  .../testing/selftests/livepatch/test-state.sh | 20 ++-----
>  7 files changed, 43 insertions(+), 89 deletions(-)
> 
> diff --git a/tools/testing/selftests/livepatch/README b/tools/testing/selftests/livepatch/README
> index 621d325425c2..0942dd5826f8 100644
> --- a/tools/testing/selftests/livepatch/README
> +++ b/tools/testing/selftests/livepatch/README
> @@ -6,8 +6,8 @@ This is a small set of sanity tests for the kernel livepatching.
>  
>  The test suite loads and unloads several test kernel modules to verify
>  livepatch behavior.  Debug information is logged to the kernel's message
> -buffer and parsed for expected messages.  (Note: the tests will clear
> -the message buffer between individual tests.)
> +buffer and parsed for expected messages.  (Note: the tests will compare
> +the message buffer for only the duration of each individual test.)
>  
>  
>  Config
> @@ -35,9 +35,9 @@ Adding tests
>  ------------
>  
>  See the common functions.sh file for the existing collection of utility
> -functions, most importantly setup_config() and check_result().  The
> -latter function greps the kernel's ring buffer for "livepatch:" and
> -"test_klp" strings, so tests be sure to include one of those strings for
> -result comparison.  Other utility functions include general module
> -loading and livepatch loading helpers (waiting for patch transitions,
> -sysfs entries, etc.)
> +functions, most importantly setup_config(), start_test() and
> +check_result().  The latter function greps the kernel's ring buffer for
> +"livepatch:" and "test_klp" strings, so tests be sure to include one of
> +those strings for result comparison.  Other utility functions include
> +general module loading and livepatch loading helpers (waiting for patch
> +transitions, sysfs entries, etc.)
> diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh
> index 2aab9791791d..e84375a33852 100644
> --- a/tools/testing/selftests/livepatch/functions.sh
> +++ b/tools/testing/selftests/livepatch/functions.sh
> @@ -243,13 +243,25 @@ function set_pre_patch_ret {
>  		die "failed to set pre_patch_ret parameter for $mod module"
>  }
>  
> +function start_test {
> +	local test="$1"
> +
> +	# Save existing dmesg so we can detect new content below
> +	SAVED_DMESG=$(mktemp --tmpdir -t klp-dmesg-XXXXXX)

There is a nice trick how to remove the temporary files even
when the script fails from other reasons. The following should
do the job:

function cleanup {
	rm -f "$SAVED_DMESG"
}

trap cleanup EXIT

> +	dmesg > "$SAVED_DMESG"
> +
> +	echo -n "TEST: $test ... "
> +}
> +
>  # check_result() - verify dmesg output
>  #	TODO - better filter, out of order msgs, etc?
>  function check_result {
>  	local expect="$*"
>  	local result
>  
> -	result=$(dmesg | grep -v 'tainting' | grep -e 'livepatch:' -e 'test_klp' | sed 's/^\[[ 0-9.]*\] //')
> +	result=$(dmesg | diff --changed-group-format='%>' --unchanged-group-format='' "$SAVED_DMESG" - | \
> +		 grep -v 'tainting' | grep -e 'livepatch:' -e 'test_klp' | \
> +		 sed 's/^\[[ 0-9.]*\] //')
>  
>  	if [[ "$expect" == "$result" ]] ; then
>  		echo "ok"
> @@ -257,4 +269,6 @@ function check_result {
>  		echo -e "not ok\n\n$(diff -upr --label expected --label result <(echo "$expect") <(echo "$result"))\n"
>  		die "livepatch kselftest(s) failed"
>  	fi
> +
> +	rm -f "$SAVED_DMESG"

This change will not be necessary with the above trap handler.

Otherwise, I really like the change. I was always a bit worried that these
tests were clearing all other messages.

Best Regards,
Petr

  parent reply	other threads:[~2020-06-12  9:49 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-10 17:20 [PATCH 0/3] selftests/livepatch: small script cleanups Joe Lawrence
2020-06-10 17:20 ` [PATCH 1/3] selftests/livepatch: Don't clear dmesg when running tests Joe Lawrence
2020-06-11  7:38   ` Miroslav Benes
2020-06-11 13:01     ` Joe Lawrence
2020-06-11 13:06       ` Miroslav Benes
2020-06-12  9:49   ` Petr Mladek [this message]
2020-06-14 15:19     ` Joe Lawrence
2020-06-12 10:11   ` Petr Mladek
2020-06-10 17:21 ` [PATCH 2/3] selftests/livepatch: use $(dmesg --notime) instead of manually filtering Joe Lawrence
2020-06-12 10:12   ` Petr Mladek
2020-06-10 17:21 ` [PATCH 3/3] selftests/livepatch: filter 'taints' from dmesg comparison Joe Lawrence
2020-06-11  7:39   ` Miroslav Benes
2020-06-11 13:10     ` Joe Lawrence
2020-06-12 11:47       ` Petr Mladek
2020-06-12 12:57         ` Kamalesh Babulal
2020-06-14 14:45           ` Joe Lawrence
2020-06-15  7:55             ` Miroslav Benes
2020-06-15  9:19               ` Petr Mladek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200612094903.GE4311@linux-b0ei \
    --to=pmladek@suse.com \
    --cc=joe.lawrence@redhat.com \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=live-patching@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).