* selftest/ftrace: Add test results to "-h" help text to ftracetest
@ 2021-06-28 13:50 Steven Rostedt
2021-06-28 14:51 ` Masami Hiramatsu
2021-06-28 15:08 ` Peter Zijlstra
0 siblings, 2 replies; 4+ messages in thread
From: Steven Rostedt @ 2021-06-28 13:50 UTC (permalink / raw)
To: Masami Hiramatsu; +Cc: LKML, Shuah Khan, Shuah Khan, Peter Zijlstra
Peter Zijlstra recently had to debug jump labels that were failing in
the ftracetest script. He became frustrated as there was no easily
available documentation on what the meanings of UNSUPPORTED and
UNRESOLVED were. The only place they are documented is in the code.
At the very least, add the results to the help text, as I find that I
also sometimes need to figure out what those terms mean.
Now "./ftracetest -h" produces:
Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]
Options:
-h|--help Show help message
-k|--keep Keep passed test logs
-v|--verbose Increase verbosity of test messages
-vv Alias of -v -v (Show all results in stdout)
-vvv Alias of -v -v -v (Show all commands immediately)
--fail-unsupported Treat UNSUPPORTED as a failure
--fail-unresolved Treat UNRESOLVED as a failure
-d|--debug Debug mode (trace all shell commands)
-l|--logdir <dir> Save logs on the <dir>
If <dir> is -, all logs output in console only
Test results:
PASS The test succeeded
FAIL The test failed, but was expected to succeed.
UNRESOLVED The test produced indeterminate results. (e.g. interrupted)
UNTESTED The test was not run, currently just a placeholder.
UNSUPPORTED The test failed because of lack of feature.
XFAIL The test failed, and was expected to fail.
Link: https://lore.kernel.org/lkml/YNmflj%2FWpFG5Ivmb@hirez.programming.kicks-ass.net/
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index 8ec1922e974e..f34580c70650 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -21,6 +21,14 @@ echo " --fail-unresolved Treat UNRESOLVED as a failure"
echo " -d|--debug Debug mode (trace all shell commands)"
echo " -l|--logdir <dir> Save logs on the <dir>"
echo " If <dir> is -, all logs output in console only"
+echo ""
+echo " Test results:"
+echo " PASS The test succeeded"
+echo " FAIL The test failed, but was expected to succeed."
+echo " UNRESOLVED The test produced indeterminate results. (e.g. interrupted)"
+echo " UNTESTED The test was not run, currently just a placeholder."
+echo " UNSUPPORTED The test failed because of lack of feature."
+echo " XFAIL The test failed, and was expected to fail."
exit $1
}
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: selftest/ftrace: Add test results to "-h" help text to ftracetest
2021-06-28 13:50 selftest/ftrace: Add test results to "-h" help text to ftracetest Steven Rostedt
@ 2021-06-28 14:51 ` Masami Hiramatsu
2021-06-28 15:08 ` Peter Zijlstra
1 sibling, 0 replies; 4+ messages in thread
From: Masami Hiramatsu @ 2021-06-28 14:51 UTC (permalink / raw)
To: Steven Rostedt; +Cc: LKML, Shuah Khan, Shuah Khan, Peter Zijlstra
On Mon, 28 Jun 2021 09:50:15 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
> Peter Zijlstra recently had to debug jump labels that were failing in
> the ftracetest script. He became frustrated as there was no easily
> available documentation on what the meanings of UNSUPPORTED and
> UNRESOLVED were. The only place they are documented is in the code.
oops, thanks for adding the help message!
>
> At the very least, add the results to the help text, as I find that I
> also sometimes need to figure out what those terms mean.
>
> Now "./ftracetest -h" produces:
>
> Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]
> Options:
> -h|--help Show help message
> -k|--keep Keep passed test logs
> -v|--verbose Increase verbosity of test messages
> -vv Alias of -v -v (Show all results in stdout)
> -vvv Alias of -v -v -v (Show all commands immediately)
> --fail-unsupported Treat UNSUPPORTED as a failure
> --fail-unresolved Treat UNRESOLVED as a failure
> -d|--debug Debug mode (trace all shell commands)
> -l|--logdir <dir> Save logs on the <dir>
> If <dir> is -, all logs output in console only
>
> Test results:
> PASS The test succeeded
> FAIL The test failed, but was expected to succeed.
> UNRESOLVED The test produced indeterminate results. (e.g. interrupted)
> UNTESTED The test was not run, currently just a placeholder.
> UNSUPPORTED The test failed because of lack of feature.
> XFAIL The test failed, and was expected to fail.
Looks good to me.
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Thank you,
>
> Link: https://lore.kernel.org/lkml/YNmflj%2FWpFG5Ivmb@hirez.programming.kicks-ass.net/
>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index 8ec1922e974e..f34580c70650 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -21,6 +21,14 @@ echo " --fail-unresolved Treat UNRESOLVED as a failure"
> echo " -d|--debug Debug mode (trace all shell commands)"
> echo " -l|--logdir <dir> Save logs on the <dir>"
> echo " If <dir> is -, all logs output in console only"
> +echo ""
> +echo " Test results:"
> +echo " PASS The test succeeded"
> +echo " FAIL The test failed, but was expected to succeed."
> +echo " UNRESOLVED The test produced indeterminate results. (e.g. interrupted)"
> +echo " UNTESTED The test was not run, currently just a placeholder."
> +echo " UNSUPPORTED The test failed because of lack of feature."
> +echo " XFAIL The test failed, and was expected to fail."
> exit $1
> }
>
--
Masami Hiramatsu <mhiramat@kernel.org>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: selftest/ftrace: Add test results to "-h" help text to ftracetest
2021-06-28 13:50 selftest/ftrace: Add test results to "-h" help text to ftracetest Steven Rostedt
2021-06-28 14:51 ` Masami Hiramatsu
@ 2021-06-28 15:08 ` Peter Zijlstra
2021-06-28 23:59 ` Masami Hiramatsu
1 sibling, 1 reply; 4+ messages in thread
From: Peter Zijlstra @ 2021-06-28 15:08 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Masami Hiramatsu, LKML, Shuah Khan, Shuah Khan
On Mon, Jun 28, 2021 at 09:50:15AM -0400, Steven Rostedt wrote:
> +echo " UNRESOLVED The test produced indeterminate results. (e.g. interrupted)"
> +echo " UNSUPPORTED The test failed because of lack of feature."
So in my case I've seen both these as a result of .config fails.
Specifically test.d/kprobe/kprobe_module.tc returns exit_unresolved when
it can't load the module (because it doesn't exist).
Also, would it be at all possible to, for both these options, always
print the missing CONFIG_symbol ? The above test actually has an echo
for it, but I've never seen the result of that appear.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: selftest/ftrace: Add test results to "-h" help text to ftracetest
2021-06-28 15:08 ` Peter Zijlstra
@ 2021-06-28 23:59 ` Masami Hiramatsu
0 siblings, 0 replies; 4+ messages in thread
From: Masami Hiramatsu @ 2021-06-28 23:59 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Steven Rostedt, Masami Hiramatsu, LKML, Shuah Khan, Shuah Khan
On Mon, 28 Jun 2021 17:08:09 +0200
Peter Zijlstra <peterz@infradead.org> wrote:
> On Mon, Jun 28, 2021 at 09:50:15AM -0400, Steven Rostedt wrote:
> > +echo " UNRESOLVED The test produced indeterminate results. (e.g. interrupted)"
> > +echo " UNSUPPORTED The test failed because of lack of feature."
>
> So in my case I've seen both these as a result of .config fails.
>
> Specifically test.d/kprobe/kprobe_module.tc returns exit_unresolved when
> it can't load the module (because it doesn't exist).
Since I defined these result code from DejaGnu (based on POSIX 1003.3) result,
I made such error (failed to load kernel module, which just gives a target
but is not an implementation of the tested feature) UNRESOLVED.
https://www.gnu.org/software/dejagnu/manual/A-POSIX-Conforming-Test-Framework.html
---
Here are some of the ways a test may wind up UNRESOLVED:
...
- The test was set up incorrectly.
---
My logic is that you are not correctly (compile and) install the target module,
so this is a setting issue.
> Also, would it be at all possible to, for both these options, always
> print the missing CONFIG_symbol ? The above test actually has an echo
> for it, but I've never seen the result of that appear.
That's a good idea :) Let me try to add depending CONFIG_ options for
each test cases.
Thank you,
--
Masami Hiramatsu <mhiramat@kernel.org>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-06-28 23:59 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-28 13:50 selftest/ftrace: Add test results to "-h" help text to ftracetest Steven Rostedt
2021-06-28 14:51 ` Masami Hiramatsu
2021-06-28 15:08 ` Peter Zijlstra
2021-06-28 23:59 ` Masami Hiramatsu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).