Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 0/2] ftrace/selftests: clean up failure cases
@ 2020-02-06 15:09 Alan Maguire
  2020-02-06 15:09 ` [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues Alan Maguire
  2020-02-06 15:09 ` [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors Alan Maguire
  0 siblings, 2 replies; 9+ messages in thread
From: Alan Maguire @ 2020-02-06 15:09 UTC (permalink / raw)
  To: rostedt, shuah, mhiramat
  Cc: mingo, linux-kselftest, linux-kernel, naveen.n.rao, colin.king

When running the ftrace selftests, 2 failures and 6 unresolved
cases were observed.  The failures can be avoided by setting
a sysctl prior to test execution (fixed in patch 1) and the
unresolved cases result from absence of testing modules which
are built based on CONFIG options being set and program
availability (fixed in patch 2).

These seem more like "unsupported" than "unresolved" errors,
since for the ftrace tests "unresolved" cases cause the test
(and thus kselftest) to report failure.  With these changes
in place, the unresolved cases become unsupported and the
test failures disappear, resulting in the ftracetest program
exiting with "ok" status.

Alan Maguire (2):
  ftrace/selftests: workaround cgroup RT scheduling issues
  ftrace/selftest: absence of modules/programs should trigger
    unsupported errors

 tools/testing/selftests/ftrace/ftracetest          | 23 ++++++++++++++++++++++
 .../ftrace/test.d/direct/ftrace-direct.tc          |  2 +-
 .../ftrace/test.d/direct/kprobe-direct.tc          |  2 +-
 .../selftests/ftrace/test.d/event/trace_printk.tc  |  2 +-
 .../ftrace/test.d/ftrace/func_mod_trace.tc         |  2 +-
 .../ftrace/test.d/kprobe/kprobe_module.tc          |  2 +-
 .../selftests/ftrace/test.d/selftest/bashisms.tc   |  2 +-
 7 files changed, 29 insertions(+), 6 deletions(-)

-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues
  2020-02-06 15:09 [PATCH 0/2] ftrace/selftests: clean up failure cases Alan Maguire
@ 2020-02-06 15:09 ` Alan Maguire
  2020-02-07  6:14   ` Masami Hiramatsu
  2020-02-06 15:09 ` [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors Alan Maguire
  1 sibling, 1 reply; 9+ messages in thread
From: Alan Maguire @ 2020-02-06 15:09 UTC (permalink / raw)
  To: rostedt, shuah, mhiramat
  Cc: mingo, linux-kselftest, linux-kernel, naveen.n.rao, colin.king

wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory
fail due to the chrt command returning:

 chrt: failed to set pid 0's policy: Operation not permitted.

To work around this, temporarily disable grout RT scheduling
during ftracetest execution.  Restore original value on
test run completion.  With these changes in place, both
tests consistently pass.

Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase")
Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase")
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index 063ecb2..3207bbf 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -29,8 +29,26 @@ err_ret=1
 # kselftest skip code is 4
 err_skip=4
 
+# cgroup RT scheduling prevents chrt commands from succeeding, which
+# induces failures in test wakeup tests.  Disable for the duration of
+# the tests.
+sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)
+
+set_sysctl() {
+  sysctl -qw ${1}=${2} >/dev/null 2>&1
+}
+
+setup() {
+  set_sysctl kernel.sched_rt_runtime_us -1
+}
+
+cleanup() {
+  set_sysctl kernel.sched_rt_runtime_us $sched_rt_runtime
+}
+
 errexit() { # message
   echo "Error: $1" 1>&2
+  cleanup
   exit $err_ret
 }
 
@@ -39,6 +57,8 @@ if [ `id -u` -ne 0 ]; then
   errexit "this must be run by root user"
 fi
 
+setup
+
 # Utilities
 absdir() { # file_path
   (cd `dirname $1`; pwd)
@@ -235,6 +255,7 @@ TOTAL_RESULT=0
 
 INSTANCE=
 CASENO=0
+
 testcase() { # testfile
   CASENO=$((CASENO+1))
   desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
@@ -406,5 +427,7 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w`
 prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w`
 prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w`
 
+cleanup
+
 # if no error, return 0
 exit $TOTAL_RESULT
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors
  2020-02-06 15:09 [PATCH 0/2] ftrace/selftests: clean up failure cases Alan Maguire
  2020-02-06 15:09 ` [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues Alan Maguire
@ 2020-02-06 15:09 ` Alan Maguire
  2020-02-07  4:43   ` Masami Hiramatsu
  1 sibling, 1 reply; 9+ messages in thread
From: Alan Maguire @ 2020-02-06 15:09 UTC (permalink / raw)
  To: rostedt, shuah, mhiramat
  Cc: mingo, linux-kselftest, linux-kernel, naveen.n.rao, colin.king

In a number of cases, the ftrace tests check for the presence of
ftrace testing-related modules (ftrace-direct, trace-printk) and
programs (checkbashisms), returning exit_unresolved if these
are not found.  The problem is, exit_unresolved causes execution
of ftracetest to return an error, when really our tests are
failing due to not having the requisite kernel configuration/tools
present, which is I think more of an unsupported error condition.
With these fixed, we see no unresolved test cases and ftracetest
returns success ("ok" when run via kselftest).

Fixes: 646f01ccdd59 ("ftrace/selftest: Add tests to test register_ftrace_direct()")
Fixes: 4d23e9b4fd2e ("selftests/ftrace: Add trace_printk sample module test")
Fixes: 7bc026d6c032 ("selftests/ftrace: Add function filter on module testcase")
Fixes: ff431b1390cb ("selftests/ftrace: Add a test to probe module functions")
Fixes: 4a075bd4e13f ("selftests/ftrace: Add checkbashisms meta-testcase")
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc  | 2 +-
 tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc  | 2 +-
 tools/testing/selftests/ftrace/test.d/event/trace_printk.tc    | 2 +-
 tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc | 2 +-
 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc  | 2 +-
 tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc     | 2 +-
 6 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
index d75a869..3d6189e 100644
--- a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
+++ b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
@@ -5,7 +5,7 @@
 rmmod ftrace-direct ||:
 if ! modprobe ftrace-direct ; then
   echo "No ftrace-direct sample module - please make CONFIG_SAMPLE_FTRACE_DIRECT=m"
-  exit_unresolved;
+  exit_unsupported;
 fi
 
 echo "Let the module run a little"
diff --git a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
index 801ecb6..3d0e3ca 100644
--- a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
+++ b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
@@ -5,7 +5,7 @@
 rmmod ftrace-direct ||:
 if ! modprobe ftrace-direct ; then
   echo "No ftrace-direct sample module - please build with CONFIG_SAMPLE_FTRACE_DIRECT=m"
-  exit_unresolved;
+  exit_unsupported;
 fi
 
 if [ ! -f kprobe_events ]; then
diff --git a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
index b02550b..dd8b10d 100644
--- a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
+++ b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
@@ -5,7 +5,7 @@
 rmmod trace-printk ||:
 if ! modprobe trace-printk ; then
   echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=m"
-  exit_unresolved;
+  exit_unsupported;
 fi
 
 echo "Waiting for irq work"
diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
index 9330c87..fc22ac0 100644
--- a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
+++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
@@ -13,7 +13,7 @@ echo '*:mod:trace_printk' > set_ftrace_filter
 if ! modprobe trace-printk ; then
   echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=
 m"
-  exit_unresolved;
+  exit_unsupported;
 fi
 
 : "Wildcard should be resolved after loading module"
diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
index d861bd7..4e07c69 100644
--- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
+++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
@@ -8,7 +8,7 @@ rmmod trace-printk ||:
 if ! modprobe trace-printk ; then
   echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=
 m"
-  exit_unresolved;
+  exit_unsupported;
 fi
 
 MOD=trace_printk
diff --git a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
index 1b081e9..1b339bd 100644
--- a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
+++ b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
@@ -9,7 +9,7 @@ fi
 
 if ! which checkbashisms > /dev/null 2>&1 ; then
   echo "No checkbashisms found. skipped."
-  exit_unresolved
+  exit_unsupported
 fi
 
 checkbashisms $FTRACETEST_ROOT/ftracetest
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors
  2020-02-06 15:09 ` [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors Alan Maguire
@ 2020-02-07  4:43   ` Masami Hiramatsu
  2020-02-07  8:27     ` Alan Maguire
  0 siblings, 1 reply; 9+ messages in thread
From: Masami Hiramatsu @ 2020-02-07  4:43 UTC (permalink / raw)
  To: Alan Maguire
  Cc: rostedt, shuah, mingo, linux-kselftest, linux-kernel,
	naveen.n.rao, colin.king

Hi Alan,

On Thu,  6 Feb 2020 15:09:20 +0000
Alan Maguire <alan.maguire@oracle.com> wrote:

> In a number of cases, the ftrace tests check for the presence of
> ftrace testing-related modules (ftrace-direct, trace-printk) and
> programs (checkbashisms), returning exit_unresolved if these
> are not found.  The problem is, exit_unresolved causes execution
> of ftracetest to return an error, when really our tests are
> failing due to not having the requisite kernel configuration/tools
> present, which is I think more of an unsupported error condition.
> With these fixed, we see no unresolved test cases and ftracetest
> returns success ("ok" when run via kselftest).

If your problem is to pass the test even if you don't test the
feature, please change the ftracetest itself instead of replacing
unresolved with unsupported. Those notice different situation.

unresolved - Testcase can not find some tools or helper drivers
             which are required for this testcase.

unsupported - Kernel does not have tested feature because of
              the version or the configuration.

Obviously the unresolved is a test environment issue. No test-module
doesn't mean no feature to be tested.
Could you tell me the reason why you can't install those required
tools and modules on the test environment?

Thank you,



> 
> Fixes: 646f01ccdd59 ("ftrace/selftest: Add tests to test register_ftrace_direct()")
> Fixes: 4d23e9b4fd2e ("selftests/ftrace: Add trace_printk sample module test")
> Fixes: 7bc026d6c032 ("selftests/ftrace: Add function filter on module testcase")
> Fixes: ff431b1390cb ("selftests/ftrace: Add a test to probe module functions")
> Fixes: 4a075bd4e13f ("selftests/ftrace: Add checkbashisms meta-testcase")
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc  | 2 +-
>  tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc  | 2 +-
>  tools/testing/selftests/ftrace/test.d/event/trace_printk.tc    | 2 +-
>  tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc | 2 +-
>  tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc  | 2 +-
>  tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc     | 2 +-
>  6 files changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
> index d75a869..3d6189e 100644
> --- a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
> +++ b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
> @@ -5,7 +5,7 @@
>  rmmod ftrace-direct ||:
>  if ! modprobe ftrace-direct ; then
>    echo "No ftrace-direct sample module - please make CONFIG_SAMPLE_FTRACE_DIRECT=m"
> -  exit_unresolved;
> +  exit_unsupported;
>  fi
>  
>  echo "Let the module run a little"
> diff --git a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
> index 801ecb6..3d0e3ca 100644
> --- a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
> +++ b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
> @@ -5,7 +5,7 @@
>  rmmod ftrace-direct ||:
>  if ! modprobe ftrace-direct ; then
>    echo "No ftrace-direct sample module - please build with CONFIG_SAMPLE_FTRACE_DIRECT=m"
> -  exit_unresolved;
> +  exit_unsupported;
>  fi
>  
>  if [ ! -f kprobe_events ]; then
> diff --git a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
> index b02550b..dd8b10d 100644
> --- a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
> +++ b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
> @@ -5,7 +5,7 @@
>  rmmod trace-printk ||:
>  if ! modprobe trace-printk ; then
>    echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=m"
> -  exit_unresolved;
> +  exit_unsupported;
>  fi
>  
>  echo "Waiting for irq work"
> diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
> index 9330c87..fc22ac0 100644
> --- a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
> +++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
> @@ -13,7 +13,7 @@ echo '*:mod:trace_printk' > set_ftrace_filter
>  if ! modprobe trace-printk ; then
>    echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=
>  m"
> -  exit_unresolved;
> +  exit_unsupported;
>  fi
>  
>  : "Wildcard should be resolved after loading module"
> diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
> index d861bd7..4e07c69 100644
> --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
> +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
> @@ -8,7 +8,7 @@ rmmod trace-printk ||:
>  if ! modprobe trace-printk ; then
>    echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=
>  m"
> -  exit_unresolved;
> +  exit_unsupported;
>  fi
>  
>  MOD=trace_printk
> diff --git a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
> index 1b081e9..1b339bd 100644
> --- a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
> +++ b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
> @@ -9,7 +9,7 @@ fi
>  
>  if ! which checkbashisms > /dev/null 2>&1 ; then
>    echo "No checkbashisms found. skipped."
> -  exit_unresolved
> +  exit_unsupported
>  fi
>  
>  checkbashisms $FTRACETEST_ROOT/ftracetest
> -- 
> 1.8.3.1
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues
  2020-02-06 15:09 ` [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues Alan Maguire
@ 2020-02-07  6:14   ` Masami Hiramatsu
  2020-02-10 22:18     ` Steven Rostedt
  0 siblings, 1 reply; 9+ messages in thread
From: Masami Hiramatsu @ 2020-02-07  6:14 UTC (permalink / raw)
  To: Alan Maguire
  Cc: rostedt, shuah, mingo, linux-kselftest, linux-kernel,
	naveen.n.rao, colin.king

On Thu,  6 Feb 2020 15:09:19 +0000
Alan Maguire <alan.maguire@oracle.com> wrote:

> wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory
> fail due to the chrt command returning:
> 
>  chrt: failed to set pid 0's policy: Operation not permitted.
> 
> To work around this, temporarily disable grout RT scheduling
> during ftracetest execution.  Restore original value on
> test run completion.  With these changes in place, both
> tests consistently pass.

OK, this looks good to me.

Acked-by: Masami Hiramatsu <mhiramat@kernel.org>

Thanks!

> 
> Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase")
> Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase")
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index 063ecb2..3207bbf 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -29,8 +29,26 @@ err_ret=1
>  # kselftest skip code is 4
>  err_skip=4
>  
> +# cgroup RT scheduling prevents chrt commands from succeeding, which
> +# induces failures in test wakeup tests.  Disable for the duration of
> +# the tests.
> +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)

OK, but can you 

> +
> +set_sysctl() {
> +  sysctl -qw ${1}=${2} >/dev/null 2>&1
> +}
> +
> +setup() {
> +  set_sysctl kernel.sched_rt_runtime_us -1
> +}
> +
> +cleanup() {
> +  set_sysctl kernel.sched_rt_runtime_us $sched_rt_runtime
> +}
> +
>  errexit() { # message
>    echo "Error: $1" 1>&2
> +  cleanup
>    exit $err_ret
>  }
>  
> @@ -39,6 +57,8 @@ if [ `id -u` -ne 0 ]; then
>    errexit "this must be run by root user"
>  fi
>  
> +setup
> +
>  # Utilities
>  absdir() { # file_path
>    (cd `dirname $1`; pwd)
> @@ -235,6 +255,7 @@ TOTAL_RESULT=0
>  
>  INSTANCE=
>  CASENO=0
> +
>  testcase() { # testfile
>    CASENO=$((CASENO+1))
>    desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
> @@ -406,5 +427,7 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w`
>  prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w`
>  prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w`
>  
> +cleanup
> +
>  # if no error, return 0
>  exit $TOTAL_RESULT
> -- 
> 1.8.3.1
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors
  2020-02-07  4:43   ` Masami Hiramatsu
@ 2020-02-07  8:27     ` Alan Maguire
  2020-02-07  8:50       ` Masami Hiramatsu
  0 siblings, 1 reply; 9+ messages in thread
From: Alan Maguire @ 2020-02-07  8:27 UTC (permalink / raw)
  To: Masami Hiramatsu
  Cc: Alan Maguire, rostedt, shuah, mingo, linux-kselftest,
	linux-kernel, naveen.n.rao, colin.king

On Fri, 7 Feb 2020, Masami Hiramatsu wrote:

> Hi Alan,
> 
> On Thu,  6 Feb 2020 15:09:20 +0000
> Alan Maguire <alan.maguire@oracle.com> wrote:
> 
> > In a number of cases, the ftrace tests check for the presence of
> > ftrace testing-related modules (ftrace-direct, trace-printk) and
> > programs (checkbashisms), returning exit_unresolved if these
> > are not found.  The problem is, exit_unresolved causes execution
> > of ftracetest to return an error, when really our tests are
> > failing due to not having the requisite kernel configuration/tools
> > present, which is I think more of an unsupported error condition.
> > With these fixed, we see no unresolved test cases and ftracetest
> > returns success ("ok" when run via kselftest).
> 
> If your problem is to pass the test even if you don't test the
> feature, please change the ftracetest itself instead of replacing
> unresolved with unsupported. Those notice different situation.
> 
> unresolved - Testcase can not find some tools or helper drivers
>              which are required for this testcase.
> 
> unsupported - Kernel does not have tested feature because of
>               the version or the configuration.
> 
> Obviously the unresolved is a test environment issue. No test-module
> doesn't mean no feature to be tested.
> Could you tell me the reason why you can't install those required
> tools and modules on the test environment?
> 

Sure! In my case, I'm testing a distro production kernel,
where I can't control the CONFIG variable settings.  In
this case, ideally I'd like the tests to return success
if no problems with ftrace were detected, even if some
of the tests could not be run due to missing modules
and programs.  As you suggest above (unless I'm
misunderstanding), this could be accomplished by modifying
ftracetest itself.  Would doing something like what is done
for UNSUPPORTED_RESULT (defaults to 0, but can be set to
1 via --fail-unsupported, such that ftracetest returns
1 if we encounter unsupported results) make sense for
the unresolved case too?

Thanks!

Alan

> Thank you,
> 
> 
> 
> > 
> > Fixes: 646f01ccdd59 ("ftrace/selftest: Add tests to test register_ftrace_direct()")
> > Fixes: 4d23e9b4fd2e ("selftests/ftrace: Add trace_printk sample module test")
> > Fixes: 7bc026d6c032 ("selftests/ftrace: Add function filter on module testcase")
> > Fixes: ff431b1390cb ("selftests/ftrace: Add a test to probe module functions")
> > Fixes: 4a075bd4e13f ("selftests/ftrace: Add checkbashisms meta-testcase")
> > Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> > ---
> >  tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc  | 2 +-
> >  tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc  | 2 +-
> >  tools/testing/selftests/ftrace/test.d/event/trace_printk.tc    | 2 +-
> >  tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc | 2 +-
> >  tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc  | 2 +-
> >  tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc     | 2 +-
> >  6 files changed, 6 insertions(+), 6 deletions(-)
> > 
> > diff --git a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
> > index d75a869..3d6189e 100644
> > --- a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc
> > @@ -5,7 +5,7 @@
> >  rmmod ftrace-direct ||:
> >  if ! modprobe ftrace-direct ; then
> >    echo "No ftrace-direct sample module - please make CONFIG_SAMPLE_FTRACE_DIRECT=m"
> > -  exit_unresolved;
> > +  exit_unsupported;
> >  fi
> >  
> >  echo "Let the module run a little"
> > diff --git a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
> > index 801ecb6..3d0e3ca 100644
> > --- a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc
> > @@ -5,7 +5,7 @@
> >  rmmod ftrace-direct ||:
> >  if ! modprobe ftrace-direct ; then
> >    echo "No ftrace-direct sample module - please build with CONFIG_SAMPLE_FTRACE_DIRECT=m"
> > -  exit_unresolved;
> > +  exit_unsupported;
> >  fi
> >  
> >  if [ ! -f kprobe_events ]; then
> > diff --git a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
> > index b02550b..dd8b10d 100644
> > --- a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
> > @@ -5,7 +5,7 @@
> >  rmmod trace-printk ||:
> >  if ! modprobe trace-printk ; then
> >    echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=m"
> > -  exit_unresolved;
> > +  exit_unsupported;
> >  fi
> >  
> >  echo "Waiting for irq work"
> > diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
> > index 9330c87..fc22ac0 100644
> > --- a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
> > @@ -13,7 +13,7 @@ echo '*:mod:trace_printk' > set_ftrace_filter
> >  if ! modprobe trace-printk ; then
> >    echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=
> >  m"
> > -  exit_unresolved;
> > +  exit_unsupported;
> >  fi
> >  
> >  : "Wildcard should be resolved after loading module"
> > diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
> > index d861bd7..4e07c69 100644
> > --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc
> > @@ -8,7 +8,7 @@ rmmod trace-printk ||:
> >  if ! modprobe trace-printk ; then
> >    echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=
> >  m"
> > -  exit_unresolved;
> > +  exit_unsupported;
> >  fi
> >  
> >  MOD=trace_printk
> > diff --git a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
> > index 1b081e9..1b339bd 100644
> > --- a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc
> > @@ -9,7 +9,7 @@ fi
> >  
> >  if ! which checkbashisms > /dev/null 2>&1 ; then
> >    echo "No checkbashisms found. skipped."
> > -  exit_unresolved
> > +  exit_unsupported
> >  fi
> >  
> >  checkbashisms $FTRACETEST_ROOT/ftracetest
> > -- 
> > 1.8.3.1
> > 
> 
> 
> -- 
> Masami Hiramatsu <mhiramat@kernel.org>
> 

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors
  2020-02-07  8:27     ` Alan Maguire
@ 2020-02-07  8:50       ` Masami Hiramatsu
  0 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu @ 2020-02-07  8:50 UTC (permalink / raw)
  To: Alan Maguire
  Cc: rostedt, shuah, mingo, linux-kselftest, linux-kernel,
	naveen.n.rao, colin.king

On Fri, 7 Feb 2020 08:27:13 +0000 (GMT)
Alan Maguire <alan.maguire@oracle.com> wrote:

> On Fri, 7 Feb 2020, Masami Hiramatsu wrote:
> 
> > Hi Alan,
> > 
> > On Thu,  6 Feb 2020 15:09:20 +0000
> > Alan Maguire <alan.maguire@oracle.com> wrote:
> > 
> > > In a number of cases, the ftrace tests check for the presence of
> > > ftrace testing-related modules (ftrace-direct, trace-printk) and
> > > programs (checkbashisms), returning exit_unresolved if these
> > > are not found.  The problem is, exit_unresolved causes execution
> > > of ftracetest to return an error, when really our tests are
> > > failing due to not having the requisite kernel configuration/tools
> > > present, which is I think more of an unsupported error condition.
> > > With these fixed, we see no unresolved test cases and ftracetest
> > > returns success ("ok" when run via kselftest).
> > 
> > If your problem is to pass the test even if you don't test the
> > feature, please change the ftracetest itself instead of replacing
> > unresolved with unsupported. Those notice different situation.
> > 
> > unresolved - Testcase can not find some tools or helper drivers
> >              which are required for this testcase.
> > 
> > unsupported - Kernel does not have tested feature because of
> >               the version or the configuration.
> > 
> > Obviously the unresolved is a test environment issue. No test-module
> > doesn't mean no feature to be tested.
> > Could you tell me the reason why you can't install those required
> > tools and modules on the test environment?
> > 
> 
> Sure! In my case, I'm testing a distro production kernel,
> where I can't control the CONFIG variable settings.  In
> this case, ideally I'd like the tests to return success
> if no problems with ftrace were detected, even if some
> of the tests could not be run due to missing modules
> and programs.

OK, for modules, we need to find another way to solve the issue.
But how about checkbashisms? you can download and build it.

https://sources.debian.org/src/devscripts/2.20.2/

For the modules, you might be able to build it from kernel
source code as out-of-tree modules, or not?
(hmm, how do the other test handle it...?)

>  As you suggest above (unless I'm
> misunderstanding), this could be accomplished by modifying
> ftracetest itself.  Would doing something like what is done
> for UNSUPPORTED_RESULT (defaults to 0, but can be set to
> 1 via --fail-unsupported, such that ftracetest returns
> 1 if we encounter unsupported results) make sense for
> the unresolved case too?

Yes, but at first could you try to setup your testing environment?
If you are officially testing your distro kernel, the distro
might need to be tested with full-set of testcases.

If not (like you are testing kernel for fun :)), you can just
make your custom set of testcases. (just remove those test files)

Thank you,


-- 
Masami Hiramatsu <mhiramat@kernel.org>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues
  2020-02-07  6:14   ` Masami Hiramatsu
@ 2020-02-10 22:18     ` Steven Rostedt
  2020-02-10 22:53       ` Masami Hiramatsu
  0 siblings, 1 reply; 9+ messages in thread
From: Steven Rostedt @ 2020-02-10 22:18 UTC (permalink / raw)
  To: Masami Hiramatsu
  Cc: Alan Maguire, shuah, mingo, linux-kselftest, linux-kernel,
	naveen.n.rao, colin.king

On Fri, 7 Feb 2020 15:14:56 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Thu,  6 Feb 2020 15:09:19 +0000
> Alan Maguire <alan.maguire@oracle.com> wrote:
> 
> > wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory
> > fail due to the chrt command returning:
> > 
> >  chrt: failed to set pid 0's policy: Operation not permitted.
> > 
> > To work around this, temporarily disable grout RT scheduling
> > during ftracetest execution.  Restore original value on
> > test run completion.  With these changes in place, both
> > tests consistently pass.  
> 
> OK, this looks good to me.
> 
> Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
> 
> Thanks!
> 
> > 
> > Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase")
> > Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase")
> > Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> > ---
> >  tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> > 
> > diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> > index 063ecb2..3207bbf 100755
> > --- a/tools/testing/selftests/ftrace/ftracetest
> > +++ b/tools/testing/selftests/ftrace/ftracetest
> > @@ -29,8 +29,26 @@ err_ret=1
> >  # kselftest skip code is 4
> >  err_skip=4
> >  
> > +# cgroup RT scheduling prevents chrt commands from succeeding, which
> > +# induces failures in test wakeup tests.  Disable for the duration of
> > +# the tests.
> > +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)  
> 
> OK, but can you 

??

Masami?

-- Steve

> 
> > +
> > +set_sysctl() {
> > +  sysctl -qw ${1}=${2} >/dev/null 2>&1
> > +}
> > +

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues
  2020-02-10 22:18     ` Steven Rostedt
@ 2020-02-10 22:53       ` Masami Hiramatsu
  0 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu @ 2020-02-10 22:53 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Alan Maguire, shuah, mingo, linux-kselftest, linux-kernel,
	naveen.n.rao, colin.king

On Mon, 10 Feb 2020 17:18:01 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Fri, 7 Feb 2020 15:14:56 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > On Thu,  6 Feb 2020 15:09:19 +0000
> > Alan Maguire <alan.maguire@oracle.com> wrote:
> > 
> > > wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory
> > > fail due to the chrt command returning:
> > > 
> > >  chrt: failed to set pid 0's policy: Operation not permitted.
> > > 
> > > To work around this, temporarily disable grout RT scheduling
> > > during ftracetest execution.  Restore original value on
> > > test run completion.  With these changes in place, both
> > > tests consistently pass.  
> > 
> > OK, this looks good to me.
> > 
> > Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
> > 
> > Thanks!
> > 
> > > 
> > > Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase")
> > > Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase")
> > > Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> > > ---
> > >  tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++
> > >  1 file changed, 23 insertions(+)
> > > 
> > > diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> > > index 063ecb2..3207bbf 100755
> > > --- a/tools/testing/selftests/ftrace/ftracetest
> > > +++ b/tools/testing/selftests/ftrace/ftracetest
> > > @@ -29,8 +29,26 @@ err_ret=1
> > >  # kselftest skip code is 4
> > >  err_skip=4
> > >  
> > > +# cgroup RT scheduling prevents chrt commands from succeeding, which
> > > +# induces failures in test wakeup tests.  Disable for the duration of
> > > +# the tests.
> > > +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)  
> > 
> > OK, but can you 
> 
> ??
> 
> Masami?

Oops, I missed to fill the comment. I meant

"but can you consider to use /proc/sys directly instead of sysctl command,
because other test cases uses /proc/sys (ftrace/fgraph-filter-stack.tc and
ftrace/func_stack_tracer.tc)?"

Thank you,

> 
> -- Steve
> 
> > 
> > > +
> > > +set_sysctl() {
> > > +  sysctl -qw ${1}=${2} >/dev/null 2>&1
> > > +}
> > > +


-- 
Masami Hiramatsu <mhiramat@kernel.org>

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, back to index

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-06 15:09 [PATCH 0/2] ftrace/selftests: clean up failure cases Alan Maguire
2020-02-06 15:09 ` [PATCH 1/2] ftrace/selftests: workaround cgroup RT scheduling issues Alan Maguire
2020-02-07  6:14   ` Masami Hiramatsu
2020-02-10 22:18     ` Steven Rostedt
2020-02-10 22:53       ` Masami Hiramatsu
2020-02-06 15:09 ` [PATCH 2/2] ftrace/selftest: absence of modules/programs should trigger unsupported errors Alan Maguire
2020-02-07  4:43   ` Masami Hiramatsu
2020-02-07  8:27     ` Alan Maguire
2020-02-07  8:50       ` Masami Hiramatsu

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git