[PATCH] selftests/ftrace: Use colored output when available

[PATCH] selftests/ftrace: Use colored output when available

[Daniel Díaz]

If test is being directly executed (with stdout opened on the
terminal) and the terminal capabilities indicate enough
colors, then use the existing scheme of green, red, and blue
to show when tests pass, fail or end in a different way.

When running the tests redirecting the stdout, for instance,
to a file, then colors are not shown, thus producing a more
readable output.

Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
---
 tools/testing/selftests/ftrace/ftracetest | 29 +++++++++++++++++------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index 4946b2edfcff..d987bbec675f 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -152,6 +152,21 @@ else
   date > $LOG_FILE
 fi
 
+# Define text colors
+# Check available colors on the terminal, if any
+ncolors=`tput colors 2>/dev/null`
+color_reset=
+color_red=
+color_green=
+color_blue=
+# If stdout exists and number of colors is eight or more, use them
+if [ -t 1 -a "$ncolors" -a "$ncolors" -ge 8 ]; then
+  color_reset="\e[0m"
+  color_red="\e[31m"
+  color_green="\e[32m"
+  color_blue="\e[34m"
+fi
+
 prlog() { # messages
   [ -z "$LOG_FILE" ] && echo -e "$@" || echo -e "$@" | tee -a $LOG_FILE
 }
@@ -195,37 +210,37 @@ test_on_instance() { # testfile
 eval_result() { # sigval
   case $1 in
     $PASS)
-      prlog "	[\e[32mPASS\e[30m]"
+      prlog "	[${color_green}PASS${color_reset}]"
       PASSED_CASES="$PASSED_CASES $CASENO"
       return 0
     ;;
     $FAIL)
-      prlog "	[\e[31mFAIL\e[30m]"
+      prlog "	[${color_red}FAIL${color_reset}]"
       FAILED_CASES="$FAILED_CASES $CASENO"
       return 1 # this is a bug.
     ;;
     $UNRESOLVED)
-      prlog "	[\e[34mUNRESOLVED\e[30m]"
+      prlog "	[${color_blue}UNRESOLVED${color_reset}]"
       UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO"
       return 1 # this is a kind of bug.. something happened.
     ;;
     $UNTESTED)
-      prlog "	[\e[34mUNTESTED\e[30m]"
+      prlog "	[${color_blue}UNTESTED${color_reset}]"
       UNTESTED_CASES="$UNTESTED_CASES $CASENO"
       return 0
     ;;
     $UNSUPPORTED)
-      prlog "	[\e[34mUNSUPPORTED\e[30m]"
+      prlog "	[${color_blue}UNSUPPORTED${color_reset}]"
       UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO"
       return $UNSUPPORTED_RESULT # depends on use case
     ;;
     $XFAIL)
-      prlog "	[\e[31mXFAIL\e[30m]"
+      prlog "	[${color_red}XFAIL${color_reset}]"
       XFAILED_CASES="$XFAILED_CASES $CASENO"
       return 0
     ;;
     *)
-      prlog "	[\e[34mUNDEFINED\e[30m]"
+      prlog "	[${color_blue}UNDEFINED${color_reset}]"
       UNDEFINED_CASES="$UNDEFINED_CASES $CASENO"
       return 1 # this must be a test bug
     ;;
-- 
2.17.1

Re: [PATCH] selftests/ftrace: Use colored output when available

[Steven Rostedt]

Masami,

Does this fix the issues you reported?

-- Steve

On Tue, 16 Oct 2018 12:02:20 -0500
Daniel Díaz <daniel.diaz@linaro.org> wrote:

> If test is being directly executed (with stdout opened on the
> terminal) and the terminal capabilities indicate enough
> colors, then use the existing scheme of green, red, and blue
> to show when tests pass, fail or end in a different way.
> 
> When running the tests redirecting the stdout, for instance,
> to a file, then colors are not shown, thus producing a more
> readable output.
> 
> Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
> ---
>  tools/testing/selftests/ftrace/ftracetest | 29 +++++++++++++++++------
>  1 file changed, 22 insertions(+), 7 deletions(-)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index 4946b2edfcff..d987bbec675f 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -152,6 +152,21 @@ else
>    date > $LOG_FILE
>  fi
>  
> +# Define text colors
> +# Check available colors on the terminal, if any
> +ncolors=`tput colors 2>/dev/null`
> +color_reset=
> +color_red=
> +color_green=
> +color_blue=
> +# If stdout exists and number of colors is eight or more, use them
> +if [ -t 1 -a "$ncolors" -a "$ncolors" -ge 8 ]; then
> +  color_reset="\e[0m"
> +  color_red="\e[31m"
> +  color_green="\e[32m"
> +  color_blue="\e[34m"
> +fi
> +
>  prlog() { # messages
>    [ -z "$LOG_FILE" ] && echo -e "$@" || echo -e "$@" | tee -a $LOG_FILE
>  }
> @@ -195,37 +210,37 @@ test_on_instance() { # testfile
>  eval_result() { # sigval
>    case $1 in
>      $PASS)
> -      prlog "	[\e[32mPASS\e[30m]"
> +      prlog "	[${color_green}PASS${color_reset}]"
>        PASSED_CASES="$PASSED_CASES $CASENO"
>        return 0
>      ;;
>      $FAIL)
> -      prlog "	[\e[31mFAIL\e[30m]"
> +      prlog "	[${color_red}FAIL${color_reset}]"
>        FAILED_CASES="$FAILED_CASES $CASENO"
>        return 1 # this is a bug.
>      ;;
>      $UNRESOLVED)
> -      prlog "	[\e[34mUNRESOLVED\e[30m]"
> +      prlog "	[${color_blue}UNRESOLVED${color_reset}]"
>        UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO"
>        return 1 # this is a kind of bug.. something happened.
>      ;;
>      $UNTESTED)
> -      prlog "	[\e[34mUNTESTED\e[30m]"
> +      prlog "	[${color_blue}UNTESTED${color_reset}]"
>        UNTESTED_CASES="$UNTESTED_CASES $CASENO"
>        return 0
>      ;;
>      $UNSUPPORTED)
> -      prlog "	[\e[34mUNSUPPORTED\e[30m]"
> +      prlog "	[${color_blue}UNSUPPORTED${color_reset}]"
>        UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO"
>        return $UNSUPPORTED_RESULT # depends on use case
>      ;;
>      $XFAIL)
> -      prlog "	[\e[31mXFAIL\e[30m]"
> +      prlog "	[${color_red}XFAIL${color_reset}]"
>        XFAILED_CASES="$XFAILED_CASES $CASENO"
>        return 0
>      ;;
>      *)
> -      prlog "	[\e[34mUNDEFINED\e[30m]"
> +      prlog "	[${color_blue}UNDEFINED${color_reset}]"
>        UNDEFINED_CASES="$UNDEFINED_CASES $CASENO"
>        return 1 # this must be a test bug
>      ;;

Re: [PATCH] selftests/ftrace: Use colored output when available

[Shuah Khan]

On 10/16/2018 11:02 AM, Daniel Díaz wrote:
> If test is being directly executed (with stdout opened on the
> terminal) and the terminal capabilities indicate enough
> colors, then use the existing scheme of green, red, and blue
> to show when tests pass, fail or end in a different way.
> 
> When running the tests redirecting the stdout, for instance,
> to a file, then colors are not shown, thus producing a more
> readable output.
> 
> Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
> ---

Hi Daniel,

Thanks for the patch. Steve is working on adding this functionality. Please
see the latest linux-kselftest next for the patch.

Steve just sent an update to the patch to fix an issue. Please test and see
if that solves the problem you are trying address with this patch.

thanks,
-- Shuah

Re: [PATCH] selftests/ftrace: Use colored output when available

[Daniel Díaz]

Hello!

On Tue, 16 Oct 2018 at 12:43, Shuah Khan <shuah@kernel.org> wrote:
> On 10/16/2018 11:02 AM, Daniel Díaz wrote:
> > If test is being directly executed (with stdout opened on the
> > terminal) and the terminal capabilities indicate enough
> > colors, then use the existing scheme of green, red, and blue
> > to show when tests pass, fail or end in a different way.
> >
> > When running the tests redirecting the stdout, for instance,
> > to a file, then colors are not shown, thus producing a more
> > readable output.
> >
> > Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
> > ---
>
> Hi Daniel,
> Thanks for the patch. Steve is working on adding this functionality. Please
> see the latest linux-kselftest next for the patch.

I hadn't seen today's discussion until after Steve replied to this
patch. Mine is based on your next branch, building upon Steve's patch.

These changes here are somewhat different, in that color is enabled
only if color is available in the terminal in the first place! And if
the standard output is redirected to a file (which is what Masami
reported today), then colors are avoided even if available.

Greetings!

Daniel Díaz
daniel.diaz@linaro.org


> Steve just sent an update to the patch to fix an issue. Please test and see
> if that solves the problem you are trying address with this patch.
>
> thanks,
> -- Shuah

Re: [PATCH] selftests/ftrace: Use colored output when available

[Steven Rostedt]

On Tue, 16 Oct 2018 11:43:29 -0600
Shuah Khan <shuah@kernel.org> wrote:

> On 10/16/2018 11:02 AM, Daniel Díaz wrote:
> > If test is being directly executed (with stdout opened on the
> > terminal) and the terminal capabilities indicate enough
> > colors, then use the existing scheme of green, red, and blue
> > to show when tests pass, fail or end in a different way.
> > 
> > When running the tests redirecting the stdout, for instance,
> > to a file, then colors are not shown, thus producing a more
> > readable output.
> > 
> > Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
> > ---  
> 
> Hi Daniel,
> 
> Thanks for the patch. Steve is working on adding this functionality. Please
> see the latest linux-kselftest next for the patch.
> 
> Steve just sent an update to the patch to fix an issue. Please test and see
> if that solves the problem you are trying address with this patch.
> 

Hi Shuah,

I think this patch actually addresses the issue of my last patch, and I
haven't sent another one.

I'm waiting to hear from Masami if it fixes the issues he saw.

-- Steve

Re: [PATCH] selftests/ftrace: Use colored output when available

[Masami Hiramatsu]

On Tue, 16 Oct 2018 13:09:56 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> 
> Masami,
> 
> Does this fix the issues you reported?

Yes, a half. We still need to strip out the escape sequences from
log file even if the terminal accepts colors.
Anyway, this fixes the "black character" issue for me :)

Thanks Daniel!

Acked-by: Masami Hiramatsu <mhiramat@kernel.org>

I'll add a filter patch on top of this.

Thank you,

> 
> -- Steve
> 
> On Tue, 16 Oct 2018 12:02:20 -0500
> Daniel Díaz <daniel.diaz@linaro.org> wrote:
> 
> > If test is being directly executed (with stdout opened on the
> > terminal) and the terminal capabilities indicate enough
> > colors, then use the existing scheme of green, red, and blue
> > to show when tests pass, fail or end in a different way.
> > 
> > When running the tests redirecting the stdout, for instance,
> > to a file, then colors are not shown, thus producing a more
> > readable output.
> > 
> > Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
> > ---
> >  tools/testing/selftests/ftrace/ftracetest | 29 +++++++++++++++++------
> >  1 file changed, 22 insertions(+), 7 deletions(-)
> > 
> > diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> > index 4946b2edfcff..d987bbec675f 100755
> > --- a/tools/testing/selftests/ftrace/ftracetest
> > +++ b/tools/testing/selftests/ftrace/ftracetest
> > @@ -152,6 +152,21 @@ else
> >    date > $LOG_FILE
> >  fi
> >  
> > +# Define text colors
> > +# Check available colors on the terminal, if any
> > +ncolors=`tput colors 2>/dev/null`
> > +color_reset=
> > +color_red=
> > +color_green=
> > +color_blue=
> > +# If stdout exists and number of colors is eight or more, use them
> > +if [ -t 1 -a "$ncolors" -a "$ncolors" -ge 8 ]; then
> > +  color_reset="\e[0m"
> > +  color_red="\e[31m"
> > +  color_green="\e[32m"
> > +  color_blue="\e[34m"
> > +fi
> > +
> >  prlog() { # messages
> >    [ -z "$LOG_FILE" ] && echo -e "$@" || echo -e "$@" | tee -a $LOG_FILE
> >  }
> > @@ -195,37 +210,37 @@ test_on_instance() { # testfile
> >  eval_result() { # sigval
> >    case $1 in
> >      $PASS)
> > -      prlog "	[\e[32mPASS\e[30m]"
> > +      prlog "	[${color_green}PASS${color_reset}]"
> >        PASSED_CASES="$PASSED_CASES $CASENO"
> >        return 0
> >      ;;
> >      $FAIL)
> > -      prlog "	[\e[31mFAIL\e[30m]"
> > +      prlog "	[${color_red}FAIL${color_reset}]"
> >        FAILED_CASES="$FAILED_CASES $CASENO"
> >        return 1 # this is a bug.
> >      ;;
> >      $UNRESOLVED)
> > -      prlog "	[\e[34mUNRESOLVED\e[30m]"
> > +      prlog "	[${color_blue}UNRESOLVED${color_reset}]"
> >        UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO"
> >        return 1 # this is a kind of bug.. something happened.
> >      ;;
> >      $UNTESTED)
> > -      prlog "	[\e[34mUNTESTED\e[30m]"
> > +      prlog "	[${color_blue}UNTESTED${color_reset}]"
> >        UNTESTED_CASES="$UNTESTED_CASES $CASENO"
> >        return 0
> >      ;;
> >      $UNSUPPORTED)
> > -      prlog "	[\e[34mUNSUPPORTED\e[30m]"
> > +      prlog "	[${color_blue}UNSUPPORTED${color_reset}]"
> >        UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO"
> >        return $UNSUPPORTED_RESULT # depends on use case
> >      ;;
> >      $XFAIL)
> > -      prlog "	[\e[31mXFAIL\e[30m]"
> > +      prlog "	[${color_red}XFAIL${color_reset}]"
> >        XFAILED_CASES="$XFAILED_CASES $CASENO"
> >        return 0
> >      ;;
> >      *)
> > -      prlog "	[\e[34mUNDEFINED\e[30m]"
> > +      prlog "	[${color_blue}UNDEFINED${color_reset}]"
> >        UNDEFINED_CASES="$UNDEFINED_CASES $CASENO"
> >        return 1 # this must be a test bug
> >      ;;
> 


-- 
Masami Hiramatsu

[PATCH] selftests/ftrace: Strip escape sequences for log file

[Masami Hiramatsu]

Strip escape sequences from the stream to the ftracetest
summary log file. Note that all test-case results are
dumped raw in each file.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
---
 tools/testing/selftests/ftrace/ftracetest |   11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index d987bbec675f..75244db70331 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -167,11 +167,18 @@ if [ -t 1 -a "$ncolors" -a "$ncolors" -ge 8 ]; then
   color_blue="\e[34m"
 fi
 
+strip_esc() {
+  # busybox sed implementation doesn't accept "\x1B", so use [:cntrl:] instead.
+  sed -E "s/[[:cntrl:]]\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g"
+}
+
 prlog() { # messages
-  [ -z "$LOG_FILE" ] && echo -e "$@" || echo -e "$@" | tee -a $LOG_FILE
+  echo -e "$@"
+  [ "$LOG_FILE" ] && echo -e "$@" | strip_esc >> $LOG_FILE
 }
 catlog() { #file
-  [ -z "$LOG_FILE" ] && cat $1 || cat $1 | tee -a $LOG_FILE
+  cat $1
+  [ "$LOG_FILE" ] && cat $1 | strip_esc >> $LOG_FILE
 }
 prlog "=== Ftrace unit tests ==="
 

Re: [PATCH] selftests/ftrace: Use colored output when available

[Shuah Khan]

On 10/16/2018 12:34 PM, Steven Rostedt wrote:
> On Tue, 16 Oct 2018 11:43:29 -0600
> Shuah Khan <shuah@kernel.org> wrote:
> 
>> On 10/16/2018 11:02 AM, Daniel Díaz wrote:
>>> If test is being directly executed (with stdout opened on the
>>> terminal) and the terminal capabilities indicate enough
>>> colors, then use the existing scheme of green, red, and blue
>>> to show when tests pass, fail or end in a different way.
>>>
>>> When running the tests redirecting the stdout, for instance,
>>> to a file, then colors are not shown, thus producing a more
>>> readable output.
>>>
>>> Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
>>> ---  
>>
>> Hi Daniel,
>>
>> Thanks for the patch. Steve is working on adding this functionality. Please
>> see the latest linux-kselftest next for the patch.
>>
>> Steve just sent an update to the patch to fix an issue. Please test and see
>> if that solves the problem you are trying address with this patch.
>>
> 
> Hi Shuah,
> 
> I think this patch actually addresses the issue of my last patch, and I
> haven't sent another one.
> 
> I'm waiting to hear from Masami if it fixes the issues he saw.
> 
> -- Steve
> 

Steve,

Do you want me to apply this patch from Daniel or do you plan to send
one. If you want to me take this one, just Ack it.

thanks,
-- Shuah

Re: [PATCH] selftests/ftrace: Use colored output when available

[Steven Rostedt]

On Tue, 16 Oct 2018 12:02:20 -0500
Daniel Díaz <daniel.diaz@linaro.org> wrote:

> If test is being directly executed (with stdout opened on the
> terminal) and the terminal capabilities indicate enough
> colors, then use the existing scheme of green, red, and blue
> to show when tests pass, fail or end in a different way.
> 
> When running the tests redirecting the stdout, for instance,
> to a file, then colors are not shown, thus producing a more
> readable output.
> 
> Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>

Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

Shuah, can you pull this into your tree?

Thanks,

-- Steve

Re: [PATCH] selftests/ftrace: Strip escape sequences for log file

[Steven Rostedt]

On Wed, 17 Oct 2018 12:33:23 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> Strip escape sequences from the stream to the ftracetest
> summary log file. Note that all test-case results are
> dumped raw in each file.
> 

Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

Shuah,

Please take this one too.

Thanks!

-- Steve

Re: [PATCH] selftests/ftrace: Use colored output when available

[Shuah Khan]

On 10/17/2018 02:41 PM, Steven Rostedt wrote:
> On Tue, 16 Oct 2018 12:02:20 -0500
> Daniel Díaz <daniel.diaz@linaro.org> wrote:
> 
>> If test is being directly executed (with stdout opened on the
>> terminal) and the terminal capabilities indicate enough
>> colors, then use the existing scheme of green, red, and blue
>> to show when tests pass, fail or end in a different way.
>>
>> When running the tests redirecting the stdout, for instance,
>> to a file, then colors are not shown, thus producing a more
>> readable output.
>>
>> Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
> 
> Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> 
> Shuah, can you pull this into your tree?
> 
Applied to linux-ksefltest next for 4.20-rc1

thanks,
-- Shuah

Re: [PATCH] selftests/ftrace: Strip escape sequences for log file

[Shuah Khan]

On 10/17/2018 02:45 PM, Steven Rostedt wrote:
> On Wed, 17 Oct 2018 12:33:23 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
>> Strip escape sequences from the stream to the ftracetest
>> summary log file. Note that all test-case results are
>> dumped raw in each file.
>>
> 
> Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> 
> Shuah,
> 
> Please take this one too.
> 

Yup. Applied to linux-kselftest for 4.20-rc1.

thanks,
-- Shuah