[PATCH v1 0/2] perf test: Output sub testing result in cs-etm

[PATCH v1 0/2] perf test: Output sub testing result in cs-etm

[Leo Yan]

The "perf test" can be integrated into testing framework, e.g. Linux
kernel functional testing (LKFT) [1].  We are not satisfied with only
outputting the summarized result for Arm CoreSight testing but lacking
more detailed result for sub testing.

This patch set is to output sub testing result in cs-etm.  Thus the
testing framework can extract the detailed info and generates reports
for which sub cases causes failure.

This patch set is cleanly applied on perf/core branch with:

  commit 6db59d357e8e ("perf arm64/s390: Fix printf conversion specifier for IP addresses")

After applied the patches, which is tested on Arm Juno-r2 board with
option '-v', the output result is shown in below; the introduced sub
testing result has the format like:

  "CoreSight path testing (CPU0 -> tmc_etf0): PASS".


  # perf test 73 -v
  73: Check Arm CoreSight trace data recording and synthesized samples:
  --- start ---
  test child forked, pid 17423
  Recording trace (only user mode) with path: CPU0 => tmc_etf0
  Looking at perf.data file for dumping branch samples:       
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:     
  CoreSight path testing (CPU0 -> tmc_etf0): PASS   
  Recording trace (only user mode) with path: CPU0 => tmc_etr0
  Looking at perf.data file for dumping branch samples:       
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:     
  CoreSight path testing (CPU0 -> tmc_etr0): PASS   
  Recording trace (only user mode) with path: CPU1 => tmc_etf0
  Looking at perf.data file for dumping branch samples:       
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:     
  CoreSight path testing (CPU1 -> tmc_etf0): PASS   
  Recording trace (only user mode) with path: CPU1 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:     
  CoreSight path testing (CPU1 -> tmc_etr0): PASS   
  Recording trace (only user mode) with path: CPU2 => tmc_etf0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:     
  CoreSight path testing (CPU2 -> tmc_etf0): PASS   
  Recording trace (only user mode) with path: CPU2 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:                  
  CoreSight path testing (CPU2 -> tmc_etr0): PASS        
  Recording trace (only user mode) with path: CPU3 => tmc_etf0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU3 -> tmc_etf0): PASS
  Recording trace (only user mode) with path: CPU3 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU3 -> tmc_etr0): PASS
  Recording trace (only user mode) with path: CPU4 => tmc_etf0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU4 -> tmc_etf0): PASS
  Recording trace (only user mode) with path: CPU4 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU4 -> tmc_etr0): PASS
  Recording trace (only user mode) with path: CPU5 => tmc_etf0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU5 -> tmc_etf0): PASS
  Recording trace (only user mode) with path: CPU5 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU5 -> tmc_etr0): PASS
  Recording trace with system wide mode
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight system wide testing: PASS
  Recording trace with snapshot mode
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight snapshot testing: PASS
  test child finished with 0
  ---- end ----
  Check Arm CoreSight trace data recording and synthesized samples: Ok

[1] https://lkft.linaro.org/


Leo Yan (2):
  perf test: Suppress logs in cs-etm testing
  perf test: Output the sub testing result in cs-etm

 tools/perf/tests/shell/test_arm_coresight.sh | 45 ++++++++++----------
 1 file changed, 23 insertions(+), 22 deletions(-)

-- 
2.25.1

[PATCH v1 1/2] perf test: Suppress logs in cs-etm testing

[Leo Yan]

With the option '-v' for the verbose logs, "perf test" outputs tons of
logs for the CoreSight case, the logs are mainly introduced by the
decoding.  And it outputs some trivial info from "perf record" command
and there have debugging info for CPU number and device name when
iterates between ETMs and sinks.

For a neat output format, this patch redirects the output logs to
"/dev/null", thus can avoid to flood logs.  And it removes the redundant
log for CPU number and device name, which have already printed out the
relevant info in the function record_touch_file().

Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 tools/perf/tests/shell/test_arm_coresight.sh | 21 +++++++++-----------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/tools/perf/tests/shell/test_arm_coresight.sh b/tools/perf/tests/shell/test_arm_coresight.sh
index 18fde2f179cd..59b647455ec6 100755
--- a/tools/perf/tests/shell/test_arm_coresight.sh
+++ b/tools/perf/tests/shell/test_arm_coresight.sh
@@ -33,7 +33,7 @@ record_touch_file() {
 	echo "Recording trace (only user mode) with path: CPU$2 => $1"
 	rm -f $file
 	perf record -o ${perfdata} -e cs_etm/@$1/u --per-thread \
-		-- taskset -c $2 touch $file
+		-- taskset -c $2 touch $file > /dev/null 2>&1
 }
 
 perf_script_branch_samples() {
@@ -43,8 +43,8 @@ perf_script_branch_samples() {
 	#   touch  6512          1         branches:u:      ffffb220824c strcmp+0xc (/lib/aarch64-linux-gnu/ld-2.27.so)
 	#   touch  6512          1         branches:u:      ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
 	#   touch  6512          1         branches:u:      ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
-	perf script -F,-time -i ${perfdata} | \
-		egrep " +$1 +[0-9]+ .* +branches:(.*:)? +"
+	perf script -F,-time -i ${perfdata} 2>&1 | \
+		egrep " +$1 +[0-9]+ .* +branches:(.*:)? +" > /dev/null 2>&1
 }
 
 perf_report_branch_samples() {
@@ -54,8 +54,8 @@ perf_report_branch_samples() {
 	#   73.04%    73.04%  touch    libc-2.27.so      [.] _dl_addr
 	#    7.71%     7.71%  touch    libc-2.27.so      [.] getenv
 	#    2.59%     2.59%  touch    ld-2.27.so        [.] strcmp
-	perf report --stdio -i ${perfdata} | \
-		egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 "
+	perf report --stdio -i ${perfdata} 2>&1 | \
+		egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 " > /dev/null 2>&1
 }
 
 perf_report_instruction_samples() {
@@ -65,8 +65,8 @@ perf_report_instruction_samples() {
 	#   68.12%  touch    libc-2.27.so   [.] _dl_addr
 	#    5.80%  touch    libc-2.27.so   [.] getenv
 	#    4.35%  touch    ld-2.27.so     [.] _dl_fixup
-	perf report --itrace=i1000i --stdio -i ${perfdata} | \
-		egrep " +[0-9]+\.[0-9]+% +$1"
+	perf report --itrace=i1000i --stdio -i ${perfdata} 2>&1 | \
+		egrep " +[0-9]+\.[0-9]+% +$1" > /dev/null 2>&1
 }
 
 is_device_sink() {
@@ -129,9 +129,6 @@ arm_cs_etm_traverse_path_test() {
 		# Find the ETM device belonging to which CPU
 		cpu=`cat $dev/cpu`
 
-		echo $dev
-		echo $cpu
-
 		# Use depth-first search (DFS) to iterate outputs
 		arm_cs_iterate_devices $dev $cpu
 	done
@@ -139,7 +136,7 @@ arm_cs_etm_traverse_path_test() {
 
 arm_cs_etm_system_wide_test() {
 	echo "Recording trace with system wide mode"
-	perf record -o ${perfdata} -e cs_etm// -a -- ls
+	perf record -o ${perfdata} -e cs_etm// -a -- ls > /dev/null 2>&1
 
 	perf_script_branch_samples perf &&
 	perf_report_branch_samples perf &&
@@ -154,7 +151,7 @@ arm_cs_etm_system_wide_test() {
 arm_cs_etm_snapshot_test() {
 	echo "Recording trace with snapshot mode"
 	perf record -o ${perfdata} -e cs_etm// -S \
-		-- dd if=/dev/zero of=/dev/null &
+		-- dd if=/dev/zero of=/dev/null > /dev/null 2>&1 &
 	PERFPID=$!
 
 	# Wait for perf program
-- 
2.25.1

[PATCH v1 2/2] perf test: Output the sub testing result in cs-etm

[Leo Yan]

The CoreSight testing contains sub cases, e.g. every CPU iterates the
possible conntected sinks and tests the paths between the associated ETM
with the found sink.  Besides the per-thread testing, it also contains
system wide testing and snapshot testing.

To easier observe results for the sub cases, this patch introduces a new
function arm_cs_report(), it outputs the result as "PASS" or "FAIL" for
every sub case; and it records the error in the variable "glb_err" which
is used as the final return value when exits the testing.

Before:

  # perf test 73 -v
  73: Check Arm CoreSight trace data recording and synthesized samples:
  --- start ---
  test child forked, pid 17423
  Recording trace (only user mode) with path: CPU0 => tmc_etf0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  Recording trace (only user mode) with path: CPU0 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:

  [...]

After:

  # perf test 73 -v
  73: Check Arm CoreSight trace data recording and synthesized samples:
  --- start ---
  test child forked, pid 17423
  Recording trace (only user mode) with path: CPU0 => tmc_etf0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU0 -> tmc_etf0): PASS
  Recording trace (only user mode) with path: CPU0 => tmc_etr0
  Looking at perf.data file for dumping branch samples:
  Looking at perf.data file for reporting branch samples:
  Looking at perf.data file for instruction samples:
  CoreSight path testing (CPU0 -> tmc_etr0): PASS
  [...]

Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 tools/perf/tests/shell/test_arm_coresight.sh | 24 ++++++++++++--------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/tools/perf/tests/shell/test_arm_coresight.sh b/tools/perf/tests/shell/test_arm_coresight.sh
index 59b647455ec6..c9eef0bba6f1 100755
--- a/tools/perf/tests/shell/test_arm_coresight.sh
+++ b/tools/perf/tests/shell/test_arm_coresight.sh
@@ -11,6 +11,7 @@
 
 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
 file=$(mktemp /tmp/temporary_file.XXXXX)
+glb_err=0
 
 skip_if_no_cs_etm_event() {
 	perf list | grep -q 'cs_etm//' && return 0
@@ -69,6 +70,15 @@ perf_report_instruction_samples() {
 		egrep " +[0-9]+\.[0-9]+% +$1" > /dev/null 2>&1
 }
 
+arm_cs_report() {
+	if [ $2 != 0 ]; then
+		echo "$1: FAIL"
+		glb_err=$2
+	else
+		echo "$1: PASS"
+	fi
+}
+
 is_device_sink() {
 	# If the node of "enable_sink" is existed under the device path, this
 	# means the device is a sink device.  Need to exclude 'tpiu' since it
@@ -113,9 +123,7 @@ arm_cs_iterate_devices() {
 			perf_report_instruction_samples touch
 
 			err=$?
-
-			# Exit when find failure
-			[ $err != 0 ] && exit $err
+			arm_cs_report "CoreSight path testing (CPU$2 -> $device_name)" $err
 		fi
 
 		arm_cs_iterate_devices $dev $2
@@ -143,9 +151,7 @@ arm_cs_etm_system_wide_test() {
 	perf_report_instruction_samples perf
 
 	err=$?
-
-	# Exit when find failure
-	[ $err != 0 ] && exit $err
+	arm_cs_report "CoreSight system wide testing" $err
 }
 
 arm_cs_etm_snapshot_test() {
@@ -169,12 +175,10 @@ arm_cs_etm_snapshot_test() {
 	perf_report_instruction_samples dd
 
 	err=$?
-
-	# Exit when find failure
-	[ $err != 0 ] && exit $err
+	arm_cs_report "CoreSight snapshot testing" $err
 }
 
 arm_cs_etm_traverse_path_test
 arm_cs_etm_system_wide_test
 arm_cs_etm_snapshot_test
-exit 0
+exit $glb_err
-- 
2.25.1

Re: [PATCH v1 0/2] perf test: Output sub testing result in cs-etm

[Arnaldo Carvalho de Melo]

Em Mon, Feb 15, 2021 at 07:59:42PM +0800, Leo Yan escreveu:
> The "perf test" can be integrated into testing framework, e.g. Linux
> kernel functional testing (LKFT) [1].  We are not satisfied with only
> outputting the summarized result for Arm CoreSight testing but lacking
> more detailed result for sub testing.
> 
> This patch set is to output sub testing result in cs-etm.  Thus the
> testing framework can extract the detailed info and generates reports
> for which sub cases causes failure.
> 
> This patch set is cleanly applied on perf/core branch with:
> 
>   commit 6db59d357e8e ("perf arm64/s390: Fix printf conversion specifier for IP addresses")
> 
> After applied the patches, which is tested on Arm Juno-r2 board with
> option '-v', the output result is shown in below; the introduced sub
> testing result has the format like:

Thanks, applied.

- Arnaldo

 
>   "CoreSight path testing (CPU0 -> tmc_etf0): PASS".
> 
> 
>   # perf test 73 -v
>   73: Check Arm CoreSight trace data recording and synthesized samples:
>   --- start ---
>   test child forked, pid 17423
>   Recording trace (only user mode) with path: CPU0 => tmc_etf0
>   Looking at perf.data file for dumping branch samples:       
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:     
>   CoreSight path testing (CPU0 -> tmc_etf0): PASS   
>   Recording trace (only user mode) with path: CPU0 => tmc_etr0
>   Looking at perf.data file for dumping branch samples:       
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:     
>   CoreSight path testing (CPU0 -> tmc_etr0): PASS   
>   Recording trace (only user mode) with path: CPU1 => tmc_etf0
>   Looking at perf.data file for dumping branch samples:       
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:     
>   CoreSight path testing (CPU1 -> tmc_etf0): PASS   
>   Recording trace (only user mode) with path: CPU1 => tmc_etr0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:     
>   CoreSight path testing (CPU1 -> tmc_etr0): PASS   
>   Recording trace (only user mode) with path: CPU2 => tmc_etf0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:     
>   CoreSight path testing (CPU2 -> tmc_etf0): PASS   
>   Recording trace (only user mode) with path: CPU2 => tmc_etr0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:                  
>   CoreSight path testing (CPU2 -> tmc_etr0): PASS        
>   Recording trace (only user mode) with path: CPU3 => tmc_etf0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight path testing (CPU3 -> tmc_etf0): PASS
>   Recording trace (only user mode) with path: CPU3 => tmc_etr0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight path testing (CPU3 -> tmc_etr0): PASS
>   Recording trace (only user mode) with path: CPU4 => tmc_etf0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight path testing (CPU4 -> tmc_etf0): PASS
>   Recording trace (only user mode) with path: CPU4 => tmc_etr0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight path testing (CPU4 -> tmc_etr0): PASS
>   Recording trace (only user mode) with path: CPU5 => tmc_etf0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight path testing (CPU5 -> tmc_etf0): PASS
>   Recording trace (only user mode) with path: CPU5 => tmc_etr0
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight path testing (CPU5 -> tmc_etr0): PASS
>   Recording trace with system wide mode
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight system wide testing: PASS
>   Recording trace with snapshot mode
>   Looking at perf.data file for dumping branch samples:
>   Looking at perf.data file for reporting branch samples:
>   Looking at perf.data file for instruction samples:
>   CoreSight snapshot testing: PASS
>   test child finished with 0
>   ---- end ----
>   Check Arm CoreSight trace data recording and synthesized samples: Ok
> 
> [1] https://lkft.linaro.org/
> 
> 
> Leo Yan (2):
>   perf test: Suppress logs in cs-etm testing
>   perf test: Output the sub testing result in cs-etm
> 
>  tools/perf/tests/shell/test_arm_coresight.sh | 45 ++++++++++----------
>  1 file changed, 23 insertions(+), 22 deletions(-)
> 
> -- 
> 2.25.1
> 

-- 

- Arnaldo