[PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Steven Rostedt]

>From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
Date: Thu, 20 Apr 2017 12:53:18 -0400
Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
 instance

An tracing instance has several of the same capabilities as the top level
instance, but may be implemented slightly different. Instead of just writing
tests that duplicat the same test cases of the top level instance, allow a
test to be written for both the top level as well as for an instance.

If a test case can be run in both the top level as well as in an tracing
instance directory, then it should add a tag "# flags: instance" in the
header of the test file. Then after all tests have run, any test that has an
instance flag set, will run again within a tracing instance.

Cc: Shuah Khan <shuah@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Suggestions-from: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..3215a8d 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -157,6 +157,10 @@ testcase() { # testfile
   prlog -n "[$CASENO]$desc"
 }
 
+test_on_instance() { # testfile
+  grep -q "^#[ \t]*flags:.*instance" $1
+}
+
 eval_result() { # sigval
   case $1 in
     $PASS)
@@ -271,6 +275,21 @@ for t in $TEST_CASES; do
   run_test $t
 done
 
+# Test on instance loop
+FIRST_INSTANCE=0
+for t in $TEST_CASES; do
+  test_on_instance $t || continue
+  if [ $FIRST_INSTANCE -eq 0 ]; then
+    FIRST_INSTANCE=1
+    echo "Running tests in a tracing instance:"
+  fi
+  SAVED_TRACING_DIR=$TRACING_DIR
+  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+  run_test $t
+  rmdir $TRACING_DIR
+  TRACING_DIR=$SAVED_TRACING_DIR
+done
+
 prlog ""
 prlog "# of passed: " `echo $PASSED_CASES | wc -w`
 prlog "# of failed: " `echo $FAILED_CASES | wc -w`
-- 
2.9.3

Re: [PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Masami Hiramatsu]

On Fri, 21 Apr 2017 23:38:50 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> Date: Thu, 20 Apr 2017 12:53:18 -0400
> Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
>  instance
> 
> An tracing instance has several of the same capabilities as the top level
> instance, but may be implemented slightly different. Instead of just writing
> tests that duplicat the same test cases of the top level instance, allow a
> test to be written for both the top level as well as for an instance.
> 
> If a test case can be run in both the top level as well as in an tracing
> instance directory, then it should add a tag "# flags: instance" in the
> header of the test file. Then after all tests have run, any test that has an
> instance flag set, will run again within a tracing instance.
> 
> Cc: Shuah Khan <shuah@kernel.org>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Suggestions-from: Masami Hiramatsu <mhiramat@kernel.org>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
>  tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index a8631d9..3215a8d 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -157,6 +157,10 @@ testcase() { # testfile
>    prlog -n "[$CASENO]$desc"
>  }
>  
> +test_on_instance() { # testfile
> +  grep -q "^#[ \t]*flags:.*instance" $1

Oops, this needs '&> /dev/null' at the end to suppress its result.

> +}
> +
>  eval_result() { # sigval
>    case $1 in
>      $PASS)
> @@ -271,6 +275,21 @@ for t in $TEST_CASES; do
>    run_test $t
>  done
>  
> +# Test on instance loop
> +FIRST_INSTANCE=0
> +for t in $TEST_CASES; do
> +  test_on_instance $t || continue
> +  if [ $FIRST_INSTANCE -eq 0 ]; then
> +    FIRST_INSTANCE=1
> +    echo "Running tests in a tracing instance:"
> +  fi

Ah, I see. This is important. And I would rather like to show
it on the description line of each test so that we can check
which test log is run in an instance. E.g. passing "(instance)"
message to run_test() and testcase() as the 2nd arg, and print
it in testlog and console?

Thank you,

> +  SAVED_TRACING_DIR=$TRACING_DIR
> +  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
> +  run_test $t
> +  rmdir $TRACING_DIR
> +  TRACING_DIR=$SAVED_TRACING_DIR
> +done
> +
>  prlog ""
>  prlog "# of passed: " `echo $PASSED_CASES | wc -w`
>  prlog "# of failed: " `echo $FAILED_CASES | wc -w`
> -- 
> 2.9.3
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Steven Rostedt]

On Sat, 22 Apr 2017 17:41:49 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Fri, 21 Apr 2017 23:38:50 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
> > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > Date: Thu, 20 Apr 2017 12:53:18 -0400
> > Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
> >  instance
> > 
> > An tracing instance has several of the same capabilities as the top level
> > instance, but may be implemented slightly different. Instead of just writing
> > tests that duplicat the same test cases of the top level instance, allow a
> > test to be written for both the top level as well as for an instance.
> > 
> > If a test case can be run in both the top level as well as in an tracing
> > instance directory, then it should add a tag "# flags: instance" in the
> > header of the test file. Then after all tests have run, any test that has an
> > instance flag set, will run again within a tracing instance.
> > 
> > Cc: Shuah Khan <shuah@kernel.org>
> > Cc: Namhyung Kim <namhyung@kernel.org>
> > Suggestions-from: Masami Hiramatsu <mhiramat@kernel.org>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> >  tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> > 
> > diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> > index a8631d9..3215a8d 100755
> > --- a/tools/testing/selftests/ftrace/ftracetest
> > +++ b/tools/testing/selftests/ftrace/ftracetest
> > @@ -157,6 +157,10 @@ testcase() { # testfile
> >    prlog -n "[$CASENO]$desc"
> >  }
> >  
> > +test_on_instance() { # testfile
> > +  grep -q "^#[ \t]*flags:.*instance" $1  
> 
> Oops, this needs '&> /dev/null' at the end to suppress its result.

The -q should keep grep from outputting anything. It does on my end. Do
you see something different?

> 
> > +}
> > +
> >  eval_result() { # sigval
> >    case $1 in
> >      $PASS)
> > @@ -271,6 +275,21 @@ for t in $TEST_CASES; do
> >    run_test $t
> >  done
> >  
> > +# Test on instance loop
> > +FIRST_INSTANCE=0
> > +for t in $TEST_CASES; do
> > +  test_on_instance $t || continue
> > +  if [ $FIRST_INSTANCE -eq 0 ]; then
> > +    FIRST_INSTANCE=1
> > +    echo "Running tests in a tracing instance:"
> > +  fi  
> 
> Ah, I see. This is important. And I would rather like to show
> it on the description line of each test so that we can check
> which test log is run in an instance. E.g. passing "(instance)"
> message to run_test() and testcase() as the 2nd arg, and print
> it in testlog and console?

OK, I'll update to v4.

Thanks,

-- Steve

> 
> Thank you,
> 
> > +  SAVED_TRACING_DIR=$TRACING_DIR
> > +  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
> > +  run_test $t
> > +  rmdir $TRACING_DIR
> > +  TRACING_DIR=$SAVED_TRACING_DIR
> > +done
> > +
> >  prlog ""
> >  prlog "# of passed: " `echo $PASSED_CASES | wc -w`
> >  prlog "# of failed: " `echo $FAILED_CASES | wc -w`
> > -- 
> > 2.9.3
> >   
> 
> 

Re: [PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Masami Hiramatsu]

On Sat, 22 Apr 2017 06:58:37 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Sat, 22 Apr 2017 17:41:49 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > On Fri, 21 Apr 2017 23:38:50 -0400
> > Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > From 4464dc867ead3ea14654165ad3ab68263aff7b17 Mon Sep 17 00:00:00 2001
> > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > Date: Thu, 20 Apr 2017 12:53:18 -0400
> > > Subject: [PATCH] selftests: ftrace: Allow some tests to be run in a tracing
> > >  instance
> > > 
> > > An tracing instance has several of the same capabilities as the top level
> > > instance, but may be implemented slightly different. Instead of just writing
> > > tests that duplicat the same test cases of the top level instance, allow a
> > > test to be written for both the top level as well as for an instance.
> > > 
> > > If a test case can be run in both the top level as well as in an tracing
> > > instance directory, then it should add a tag "# flags: instance" in the
> > > header of the test file. Then after all tests have run, any test that has an
> > > instance flag set, will run again within a tracing instance.
> > > 
> > > Cc: Shuah Khan <shuah@kernel.org>
> > > Cc: Namhyung Kim <namhyung@kernel.org>
> > > Suggestions-from: Masami Hiramatsu <mhiramat@kernel.org>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > >  tools/testing/selftests/ftrace/ftracetest | 19 +++++++++++++++++++
> > >  1 file changed, 19 insertions(+)
> > > 
> > > diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> > > index a8631d9..3215a8d 100755
> > > --- a/tools/testing/selftests/ftrace/ftracetest
> > > +++ b/tools/testing/selftests/ftrace/ftracetest
> > > @@ -157,6 +157,10 @@ testcase() { # testfile
> > >    prlog -n "[$CASENO]$desc"
> > >  }
> > >  
> > > +test_on_instance() { # testfile
> > > +  grep -q "^#[ \t]*flags:.*instance" $1  
> > 
> > Oops, this needs '&> /dev/null' at the end to suppress its result.
> 
> The -q should keep grep from outputting anything. It does on my end. Do
> you see something different?

Ah, sorry I missed -q option... OK, it is good!

> 
> > 
> > > +}
> > > +
> > >  eval_result() { # sigval
> > >    case $1 in
> > >      $PASS)
> > > @@ -271,6 +275,21 @@ for t in $TEST_CASES; do
> > >    run_test $t
> > >  done
> > >  
> > > +# Test on instance loop
> > > +FIRST_INSTANCE=0
> > > +for t in $TEST_CASES; do
> > > +  test_on_instance $t || continue
> > > +  if [ $FIRST_INSTANCE -eq 0 ]; then
> > > +    FIRST_INSTANCE=1
> > > +    echo "Running tests in a tracing instance:"
> > > +  fi  
> > 
> > Ah, I see. This is important. And I would rather like to show
> > it on the description line of each test so that we can check
> > which test log is run in an instance. E.g. passing "(instance)"
> > message to run_test() and testcase() as the 2nd arg, and print
> > it in testlog and console?
> 
> OK, I'll update to v4.

Thank you!

> 
> Thanks,
> 
> -- Steve
> 
> > 
> > Thank you,
> > 
> > > +  SAVED_TRACING_DIR=$TRACING_DIR
> > > +  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
> > > +  run_test $t
> > > +  rmdir $TRACING_DIR
> > > +  TRACING_DIR=$SAVED_TRACING_DIR
> > > +done
> > > +
> > >  prlog ""
> > >  prlog "# of passed: " `echo $PASSED_CASES | wc -w`
> > >  prlog "# of failed: " `echo $FAILED_CASES | wc -w`
> > > -- 
> > > 2.9.3
> > >   
> > 
> > 
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Steven Rostedt]

On Sun, 23 Apr 2017 09:02:24 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:


> > >   
> > > > +}
> > > > +
> > > >  eval_result() { # sigval
> > > >    case $1 in
> > > >      $PASS)
> > > > @@ -271,6 +275,21 @@ for t in $TEST_CASES; do
> > > >    run_test $t
> > > >  done
> > > >  
> > > > +# Test on instance loop
> > > > +FIRST_INSTANCE=0
> > > > +for t in $TEST_CASES; do
> > > > +  test_on_instance $t || continue
> > > > +  if [ $FIRST_INSTANCE -eq 0 ]; then
> > > > +    FIRST_INSTANCE=1
> > > > +    echo "Running tests in a tracing instance:"
> > > > +  fi    
> > > 
> > > Ah, I see. This is important. And I would rather like to show
> > > it on the description line of each test so that we can check
> > > which test log is run in an instance. E.g. passing "(instance)"
> > > message to run_test() and testcase() as the 2nd arg, and print
> > > it in testlog and console?  
> > 
> > OK, I'll update to v4.  

What about something like this:

-- Steve

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index a8631d9..32e6211 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -150,11 +150,16 @@ XFAILED_CASES=
 UNDEFINED_CASES=
 TOTAL_RESULT=0
 
+INSTANCE=
 CASENO=0
 testcase() { # testfile
   CASENO=$((CASENO+1))
   desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
-  prlog -n "[$CASENO]$desc"
+  prlog -n "[$CASENO]$INSTANCE$desc"
+}
+
+test_on_instance() { # testfile
+  grep -q "^#[ \t]*flags:.*instance" $1
 }
 
 eval_result() { # sigval
@@ -271,6 +276,17 @@ for t in $TEST_CASES; do
   run_test $t
 done
 
+# Test on instance loop
+INSTANCE=" (instance) "
+for t in $TEST_CASES; do
+  test_on_instance $t || continue
+  SAVED_TRACING_DIR=$TRACING_DIR
+  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+  run_test $t
+  rmdir $TRACING_DIR
+  TRACING_DIR=$SAVED_TRACING_DIR
+done
+
 prlog ""
 prlog "# of passed: " `echo $PASSED_CASES | wc -w`
 prlog "# of failed: " `echo $FAILED_CASES | wc -w`

Re: [PATCH v3] selftests: ftrace: Allow some tests to be run in a tracing instance

[Masami Hiramatsu]

On Mon, 24 Apr 2017 18:01:38 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Sun, 23 Apr 2017 09:02:24 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> 
> > > >   
> > > > > +}
> > > > > +
> > > > >  eval_result() { # sigval
> > > > >    case $1 in
> > > > >      $PASS)
> > > > > @@ -271,6 +275,21 @@ for t in $TEST_CASES; do
> > > > >    run_test $t
> > > > >  done
> > > > >  
> > > > > +# Test on instance loop
> > > > > +FIRST_INSTANCE=0
> > > > > +for t in $TEST_CASES; do
> > > > > +  test_on_instance $t || continue
> > > > > +  if [ $FIRST_INSTANCE -eq 0 ]; then
> > > > > +    FIRST_INSTANCE=1
> > > > > +    echo "Running tests in a tracing instance:"
> > > > > +  fi    
> > > > 
> > > > Ah, I see. This is important. And I would rather like to show
> > > > it on the description line of each test so that we can check
> > > > which test log is run in an instance. E.g. passing "(instance)"
> > > > message to run_test() and testcase() as the 2nd arg, and print
> > > > it in testlog and console?  
> > > 
> > > OK, I'll update to v4.  
> 
> What about something like this:

Ah, that's much better than I thought :)

Acked-by: Masami Hiramatsu <mhiramat@kernel.org>

Thank you!

> 
> -- Steve
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index a8631d9..32e6211 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -150,11 +150,16 @@ XFAILED_CASES=
>  UNDEFINED_CASES=
>  TOTAL_RESULT=0
>  
> +INSTANCE=
>  CASENO=0
>  testcase() { # testfile
>    CASENO=$((CASENO+1))
>    desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:`
> -  prlog -n "[$CASENO]$desc"
> +  prlog -n "[$CASENO]$INSTANCE$desc"
> +}
> +
> +test_on_instance() { # testfile
> +  grep -q "^#[ \t]*flags:.*instance" $1
>  }
>  
>  eval_result() { # sigval
> @@ -271,6 +276,17 @@ for t in $TEST_CASES; do
>    run_test $t
>  done
>  
> +# Test on instance loop
> +INSTANCE=" (instance) "
> +for t in $TEST_CASES; do
> +  test_on_instance $t || continue
> +  SAVED_TRACING_DIR=$TRACING_DIR
> +  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
> +  run_test $t
> +  rmdir $TRACING_DIR
> +  TRACING_DIR=$SAVED_TRACING_DIR
> +done
> +
>  prlog ""
>  prlog "# of passed: " `echo $PASSED_CASES | wc -w`
>  prlog "# of failed: " `echo $FAILED_CASES | wc -w`


-- 
Masami Hiramatsu <mhiramat@kernel.org>