[PATCH v4] perf test: Introduce script for Arm CoreSight testing

[PATCH v4] perf test: Introduce script for Arm CoreSight testing

[Leo Yan]

We need a simple method to test Perf with Arm CoreSight drivers, this
could be used for smoke testing when new patch is coming for perf or
CoreSight drivers, and we also can use the test to confirm if the
CoreSight has been enabled successfully on new platforms.

This patch introduces the shell script test_arm_coresight.sh which is
under the 'pert test' framework.  This script provides three testing
scenarios:

Test scenario 1: traverse all possible paths between source and sink

For traversing possible paths, simply to say, the testing rationale
is source oriented testing, it traverses every source (now only refers
to ETM device) and test its all possible sinks.  To search the complete
paths from one specific source to its sinks, this patch relies on the
sysfs '/sys/bus/coresight/devices/devX/out:Y' for depth-first search
(DFS) for iteration connected device nodes, if the output device is
detected as one of ETR, ETF, or ETB types then it will test trace data
recording and decoding for this PMU device.

The script runs three output testings for every trace data:
- Test branch samples dumping with 'perf script' command;
- Test branch samples reporting with 'perf report' command;
- Use option '--itrace=i1000i' to insert synthesized instructions events
  and the script will check if perf can output the percentage value
  successfully based on the instruction samples.

Test scenario 2: CPU wide mode test

For CPU wide mode testing, it passes option '-a' to perf tool to enable
tracing on all CPUs, so it's hard to say which program will be traced.
But perf tool itself contributes much overload in this case, so it will
parse trace data and check if process 'perf' can be detected or not.

Test scenario 3: snapshot mode test.

For snapshot mode testing, it uses 'dd' command to launch a long running
program, so this can give chance to send signal -USR2; it will check the
captured trace data contains 'dd' related thread info or not.

If any test fails, it will report failure and directly exit with error.
This test will be only applied on a platform with PMU event 'cs_etm//',
otherwise will skip the testing.

Below is detailed usage for it:

  # cd $linux/tools/perf  -> This is important so can use shell script
  # perf test list
    [...]
    65: probe libc's inet_pton & backtrace it with ping
    66: Check Arm CoreSight trace data recording and branch samples
    67: Check open filename arg using perf trace + vfs_getname
    68: Zstd perf.data compression/decompression
    69: Add vfs_getname probe to get syscall args filenames
    70: Use vfs_getname probe to get syscall args filenames

  # perf test 66
    66: Check Arm CoreSight trace data recording and branch samples: Ok

Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 tools/perf/tests/shell/test_arm_coresight.sh | 172 +++++++++++++++++++
 1 file changed, 172 insertions(+)
 create mode 100755 tools/perf/tests/shell/test_arm_coresight.sh

diff --git a/tools/perf/tests/shell/test_arm_coresight.sh b/tools/perf/tests/shell/test_arm_coresight.sh
new file mode 100755
index 000000000000..73b973bada26
--- /dev/null
+++ b/tools/perf/tests/shell/test_arm_coresight.sh
@@ -0,0 +1,172 @@
+#!/bin/sh
+# Check Arm CoreSight trace data recording and branch samples
+
+# Uses the 'perf record' to record trace data with Arm CoreSight sinks;
+# then verify if there have any branch samples and instruction samples
+# are generated by CoreSight with 'perf script' and 'perf report'
+# commands.
+
+# SPDX-License-Identifier: GPL-2.0
+# Leo Yan <leo.yan@linaro.org>, 2020
+
+perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+file=$(mktemp /tmp/temporary_file.XXXXX)
+
+skip_if_no_cs_etm_event() {
+	perf list | grep -q 'cs_etm//' && return 0
+
+	# cs_etm event doesn't exist
+	return 2
+}
+
+skip_if_no_cs_etm_event || exit 2
+
+record_touch_file() {
+	echo "Recording trace (only user mode) with path: CPU$2 => $1"
+	perf record -o ${perfdata} -e cs_etm/@$1/u --per-thread \
+		-- taskset -c $2 touch $file
+}
+
+perf_script_branch_samples() {
+	echo "Looking at perf.data file for dumping branch samples:"
+
+	# Below is an example of the branch samples dumping:
+	#   touch  6512          1         branches:u:      ffffb220824c strcmp+0xc (/lib/aarch64-linux-gnu/ld-2.27.so)
+	#   touch  6512          1         branches:u:      ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
+	#   touch  6512          1         branches:u:      ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
+	perf script -F,-time -i ${perfdata} | \
+		egrep " +$1 +[0-9]+ .* +branches:([u|k]:)? +"
+}
+
+perf_report_branch_samples() {
+	echo "Looking at perf.data file for reporting branch samples:"
+
+	# Below is an example of the branch samples reporting:
+	#   73.04%    73.04%  touch    libc-2.27.so      [.] _dl_addr
+	#    7.71%     7.71%  touch    libc-2.27.so      [.] getenv
+	#    2.59%     2.59%  touch    ld-2.27.so        [.] strcmp
+	perf report --stdio -i ${perfdata} | \
+		egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 "
+}
+
+perf_report_instruction_samples() {
+	echo "Looking at perf.data file for instruction samples:"
+
+	# Below is an example of the instruction samples reporting:
+	#   68.12%  touch    libc-2.27.so   [.] _dl_addr
+	#    5.80%  touch    libc-2.27.so   [.] getenv
+	#    4.35%  touch    ld-2.27.so     [.] _dl_fixup
+	perf report --itrace=i1000i --stdio -i ${perfdata} | \
+		egrep " +[0-9]+\.[0-9]+% +$1"
+}
+
+arm_cs_iterate_devices() {
+	for dev in $1/connections/out\:*; do
+
+		# Skip testing if it's not a directory
+		! [ -d $dev ] && continue;
+
+		# Read out its symbol link file name
+		path=`readlink -f $dev`
+
+		# Extract device name from path, e.g.
+		#   path = '/sys/devices/platform/20010000.etf/tmc_etf0'
+		#     `> device_name = 'tmc_etf0'
+		device_name=`echo $path | awk -F/ '{print $(NF)}'`
+
+		echo $device_name | egrep -q "etr|etb|etf"
+
+		# Only test if the output device is ETR/ETB/ETF
+		if [ $? -eq 0 ]; then
+
+			pmu_dev="/sys/bus/event_source/devices/cs_etm/sinks/$device_name"
+
+			# Exit if PMU device node doesn't exist
+			if ! [ -f $pmu_dev ]; then
+				echo "PMU device $pmu_dev doesn't exist"
+				exit 1
+			fi
+
+			record_touch_file $device_name $2 &&
+				perf_script_branch_samples touch &&
+				perf_report_branch_samples touch &&
+				perf_report_instruction_samples touch
+
+			err=$?
+
+			# Exit when find failure
+			[ $err != 0 ] && exit $err
+
+			rm -f ${perfdata}
+			rm -f ${file}
+		fi
+
+		arm_cs_iterate_devices $dev $2
+	done
+}
+
+arm_cs_etm_traverse_path_test() {
+	# Iterate for every ETM device
+	for dev in /sys/bus/coresight/devices/etm*; do
+
+		# Find the ETM device belonging to which CPU
+		cpu=`cat $dev/cpu`
+
+		echo $dev
+		echo $cpu
+
+		# Use depth-first search (DFS) to iterate outputs
+		arm_cs_iterate_devices $dev $cpu
+	done
+}
+
+arm_cs_etm_cpu_wide_test() {
+	echo "Recording trace with CPU wide mode"
+	perf record -o ${perfdata} -e cs_etm// -a -- ls
+
+	perf_script_branch_samples perf &&
+	perf_report_branch_samples perf &&
+	perf_report_instruction_samples perf
+
+	err=$?
+
+	# Exit when find failure
+	[ $err != 0 ] && exit $err
+
+	rm -f ${perfdata}
+	rm -f ${file}
+}
+
+arm_cs_etm_snapshot_test() {
+	echo "Recording trace with snapshot mode"
+	perf record -o ${perfdata} -e cs_etm// -S --per-thread \
+		-- dd if=/dev/zero of=/dev/null &
+	PERFPID=$!
+
+	# Wait for perf program
+	sleep 1
+
+	# Send signal to snapshot trace data
+	kill -USR2 $PERFPID
+
+	# Stop perf program
+	kill $PERFPID
+	wait $PERFPID
+
+	perf_script_branch_samples dd &&
+	perf_report_branch_samples dd &&
+	perf_report_instruction_samples dd
+
+	err=$?
+
+	# Exit when find failure
+	[ $err != 0 ] && exit $err
+
+	rm -f ${perfdata}
+	rm -f ${file}
+}
+
+arm_cs_etm_traverse_path_test
+arm_cs_etm_cpu_wide_test
+arm_cs_etm_snapshot_test
+exit 0
-- 
2.17.1

Re: [PATCH v4] perf test: Introduce script for Arm CoreSight testing

[Mathieu Poirier]

Leo and Arnaldo,

On Thu, 6 Aug 2020 at 01:02, Leo Yan <leo.yan@linaro.org> wrote:
>
> We need a simple method to test Perf with Arm CoreSight drivers, this
> could be used for smoke testing when new patch is coming for perf or
> CoreSight drivers, and we also can use the test to confirm if the
> CoreSight has been enabled successfully on new platforms.
>

I have a lot of patches to review - getting to this one will take a while.

Thanks for the patience,
Mathieu

> This patch introduces the shell script test_arm_coresight.sh which is
> under the 'pert test' framework.  This script provides three testing
> scenarios:
>
> Test scenario 1: traverse all possible paths between source and sink
>
> For traversing possible paths, simply to say, the testing rationale
> is source oriented testing, it traverses every source (now only refers
> to ETM device) and test its all possible sinks.  To search the complete
> paths from one specific source to its sinks, this patch relies on the
> sysfs '/sys/bus/coresight/devices/devX/out:Y' for depth-first search
> (DFS) for iteration connected device nodes, if the output device is
> detected as one of ETR, ETF, or ETB types then it will test trace data
> recording and decoding for this PMU device.
>
> The script runs three output testings for every trace data:
> - Test branch samples dumping with 'perf script' command;
> - Test branch samples reporting with 'perf report' command;
> - Use option '--itrace=i1000i' to insert synthesized instructions events
>   and the script will check if perf can output the percentage value
>   successfully based on the instruction samples.
>
> Test scenario 2: CPU wide mode test
>
> For CPU wide mode testing, it passes option '-a' to perf tool to enable
> tracing on all CPUs, so it's hard to say which program will be traced.
> But perf tool itself contributes much overload in this case, so it will
> parse trace data and check if process 'perf' can be detected or not.
>
> Test scenario 3: snapshot mode test.
>
> For snapshot mode testing, it uses 'dd' command to launch a long running
> program, so this can give chance to send signal -USR2; it will check the
> captured trace data contains 'dd' related thread info or not.
>
> If any test fails, it will report failure and directly exit with error.
> This test will be only applied on a platform with PMU event 'cs_etm//',
> otherwise will skip the testing.
>
> Below is detailed usage for it:
>
>   # cd $linux/tools/perf  -> This is important so can use shell script
>   # perf test list
>     [...]
>     65: probe libc's inet_pton & backtrace it with ping
>     66: Check Arm CoreSight trace data recording and branch samples
>     67: Check open filename arg using perf trace + vfs_getname
>     68: Zstd perf.data compression/decompression
>     69: Add vfs_getname probe to get syscall args filenames
>     70: Use vfs_getname probe to get syscall args filenames
>
>   # perf test 66
>     66: Check Arm CoreSight trace data recording and branch samples: Ok
>
> Signed-off-by: Leo Yan <leo.yan@linaro.org>
> ---
>  tools/perf/tests/shell/test_arm_coresight.sh | 172 +++++++++++++++++++
>  1 file changed, 172 insertions(+)
>  create mode 100755 tools/perf/tests/shell/test_arm_coresight.sh
>
> diff --git a/tools/perf/tests/shell/test_arm_coresight.sh b/tools/perf/tests/shell/test_arm_coresight.sh
> new file mode 100755
> index 000000000000..73b973bada26
> --- /dev/null
> +++ b/tools/perf/tests/shell/test_arm_coresight.sh
> @@ -0,0 +1,172 @@
> +#!/bin/sh
> +# Check Arm CoreSight trace data recording and branch samples
> +
> +# Uses the 'perf record' to record trace data with Arm CoreSight sinks;
> +# then verify if there have any branch samples and instruction samples
> +# are generated by CoreSight with 'perf script' and 'perf report'
> +# commands.
> +
> +# SPDX-License-Identifier: GPL-2.0
> +# Leo Yan <leo.yan@linaro.org>, 2020
> +
> +perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
> +file=$(mktemp /tmp/temporary_file.XXXXX)
> +
> +skip_if_no_cs_etm_event() {
> +       perf list | grep -q 'cs_etm//' && return 0
> +
> +       # cs_etm event doesn't exist
> +       return 2
> +}
> +
> +skip_if_no_cs_etm_event || exit 2
> +
> +record_touch_file() {
> +       echo "Recording trace (only user mode) with path: CPU$2 => $1"
> +       perf record -o ${perfdata} -e cs_etm/@$1/u --per-thread \
> +               -- taskset -c $2 touch $file
> +}
> +
> +perf_script_branch_samples() {
> +       echo "Looking at perf.data file for dumping branch samples:"
> +
> +       # Below is an example of the branch samples dumping:
> +       #   touch  6512          1         branches:u:      ffffb220824c strcmp+0xc (/lib/aarch64-linux-gnu/ld-2.27.so)
> +       #   touch  6512          1         branches:u:      ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
> +       #   touch  6512          1         branches:u:      ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
> +       perf script -F,-time -i ${perfdata} | \
> +               egrep " +$1 +[0-9]+ .* +branches:([u|k]:)? +"
> +}
> +
> +perf_report_branch_samples() {
> +       echo "Looking at perf.data file for reporting branch samples:"
> +
> +       # Below is an example of the branch samples reporting:
> +       #   73.04%    73.04%  touch    libc-2.27.so      [.] _dl_addr
> +       #    7.71%     7.71%  touch    libc-2.27.so      [.] getenv
> +       #    2.59%     2.59%  touch    ld-2.27.so        [.] strcmp
> +       perf report --stdio -i ${perfdata} | \
> +               egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 "
> +}
> +
> +perf_report_instruction_samples() {
> +       echo "Looking at perf.data file for instruction samples:"
> +
> +       # Below is an example of the instruction samples reporting:
> +       #   68.12%  touch    libc-2.27.so   [.] _dl_addr
> +       #    5.80%  touch    libc-2.27.so   [.] getenv
> +       #    4.35%  touch    ld-2.27.so     [.] _dl_fixup
> +       perf report --itrace=i1000i --stdio -i ${perfdata} | \
> +               egrep " +[0-9]+\.[0-9]+% +$1"
> +}
> +
> +arm_cs_iterate_devices() {
> +       for dev in $1/connections/out\:*; do
> +
> +               # Skip testing if it's not a directory
> +               ! [ -d $dev ] && continue;
> +
> +               # Read out its symbol link file name
> +               path=`readlink -f $dev`
> +
> +               # Extract device name from path, e.g.
> +               #   path = '/sys/devices/platform/20010000.etf/tmc_etf0'
> +               #     `> device_name = 'tmc_etf0'
> +               device_name=`echo $path | awk -F/ '{print $(NF)}'`
> +
> +               echo $device_name | egrep -q "etr|etb|etf"
> +
> +               # Only test if the output device is ETR/ETB/ETF
> +               if [ $? -eq 0 ]; then
> +
> +                       pmu_dev="/sys/bus/event_source/devices/cs_etm/sinks/$device_name"
> +
> +                       # Exit if PMU device node doesn't exist
> +                       if ! [ -f $pmu_dev ]; then
> +                               echo "PMU device $pmu_dev doesn't exist"
> +                               exit 1
> +                       fi
> +
> +                       record_touch_file $device_name $2 &&
> +                               perf_script_branch_samples touch &&
> +                               perf_report_branch_samples touch &&
> +                               perf_report_instruction_samples touch
> +
> +                       err=$?
> +
> +                       # Exit when find failure
> +                       [ $err != 0 ] && exit $err
> +
> +                       rm -f ${perfdata}
> +                       rm -f ${file}
> +               fi
> +
> +               arm_cs_iterate_devices $dev $2
> +       done
> +}
> +
> +arm_cs_etm_traverse_path_test() {
> +       # Iterate for every ETM device
> +       for dev in /sys/bus/coresight/devices/etm*; do
> +
> +               # Find the ETM device belonging to which CPU
> +               cpu=`cat $dev/cpu`
> +
> +               echo $dev
> +               echo $cpu
> +
> +               # Use depth-first search (DFS) to iterate outputs
> +               arm_cs_iterate_devices $dev $cpu
> +       done
> +}
> +
> +arm_cs_etm_cpu_wide_test() {
> +       echo "Recording trace with CPU wide mode"
> +       perf record -o ${perfdata} -e cs_etm// -a -- ls
> +
> +       perf_script_branch_samples perf &&
> +       perf_report_branch_samples perf &&
> +       perf_report_instruction_samples perf
> +
> +       err=$?
> +
> +       # Exit when find failure
> +       [ $err != 0 ] && exit $err
> +
> +       rm -f ${perfdata}
> +       rm -f ${file}
> +}
> +
> +arm_cs_etm_snapshot_test() {
> +       echo "Recording trace with snapshot mode"
> +       perf record -o ${perfdata} -e cs_etm// -S --per-thread \
> +               -- dd if=/dev/zero of=/dev/null &
> +       PERFPID=$!
> +
> +       # Wait for perf program
> +       sleep 1
> +
> +       # Send signal to snapshot trace data
> +       kill -USR2 $PERFPID
> +
> +       # Stop perf program
> +       kill $PERFPID
> +       wait $PERFPID
> +
> +       perf_script_branch_samples dd &&
> +       perf_report_branch_samples dd &&
> +       perf_report_instruction_samples dd
> +
> +       err=$?
> +
> +       # Exit when find failure
> +       [ $err != 0 ] && exit $err
> +
> +       rm -f ${perfdata}
> +       rm -f ${file}
> +}
> +
> +arm_cs_etm_traverse_path_test
> +arm_cs_etm_cpu_wide_test
> +arm_cs_etm_snapshot_test
> +exit 0
> --
> 2.17.1
>

Re: [PATCH v4] perf test: Introduce script for Arm CoreSight testing

[Suzuki K Poulose]

Hi Leo,

On 08/06/2020 08:02 AM, Leo Yan wrote:
> We need a simple method to test Perf with Arm CoreSight drivers, this
> could be used for smoke testing when new patch is coming for perf or
> CoreSight drivers, and we also can use the test to confirm if the
> CoreSight has been enabled successfully on new platforms.
> 
> This patch introduces the shell script test_arm_coresight.sh which is
> under the 'pert test' framework.  This script provides three testing
> scenarios:

Thank you for this testcase. It is a very good tool for people
check their system for CoreSight driver functionality.

> 
> Test scenario 1: traverse all possible paths between source and sink
> 
> For traversing possible paths, simply to say, the testing rationale
> is source oriented testing, it traverses every source (now only refers
> to ETM device) and test its all possible sinks.  To search the complete
> paths from one specific source to its sinks, this patch relies on the
> sysfs '/sys/bus/coresight/devices/devX/out:Y' for depth-first search
> (DFS) for iteration connected device nodes, if the output device is
> detected as one of ETR, ETF, or ETB types then it will test trace data


Please see my suggestion below, to use "enable_sink" as an indicator
for a sink device.

> recording and decoding for this PMU device.
> 
> The script runs three output testings for every trace data:
> - Test branch samples dumping with 'perf script' command;
> - Test branch samples reporting with 'perf report' command;
> - Use option '--itrace=i1000i' to insert synthesized instructions events
>    and the script will check if perf can output the percentage value
>    successfully based on the instruction samples.
> 
> Test scenario 2: CPU wide mode test
>  > For CPU wide mode testing, it passes option '-a' to perf tool to enable
> tracing on all CPUs, so it's hard to say which program will be traced.

Isn't this system-wide, when you trace all CPUs ? In CPU wide mode,
you specify a list of CPUs (-C ?). I always get confused here.

> But perf tool itself contributes much overload in this case, so it will
> parse trace data and check if process 'perf' can be detected or not.
> 
> diff --git a/tools/perf/tests/shell/test_arm_coresight.sh b/tools/perf/tests/shell/test_arm_coresight.sh
> new file mode 100755
> index 000000000000..73b973bada26
> --- /dev/null
> +++ b/tools/perf/tests/shell/test_arm_coresight.sh
> @@ -0,0 +1,172 @@
> +#!/bin/sh
> +# Check Arm CoreSight trace data recording and branch samples
> +
> +# Uses the 'perf record' to record trace data with Arm CoreSight sinks;
> +# then verify if there have any branch samples and instruction samples
> +# are generated by CoreSight with 'perf script' and 'perf report'
> +# commands.
> +
> +# SPDX-License-Identifier: GPL-2.0
> +# Leo Yan <leo.yan@linaro.org>, 2020
> +
> +perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
> +file=$(mktemp /tmp/temporary_file.XXXXX)
> +
> +skip_if_no_cs_etm_event() {
> +	perf list | grep -q 'cs_etm//' && return 0
> +
> +	# cs_etm event doesn't exist
> +	return 2
> +}
> +
> +skip_if_no_cs_etm_event || exit 2
> +
> +record_touch_file() {
> +	echo "Recording trace (only user mode) with path: CPU$2 => $1"
> +	perf record -o ${perfdata} -e cs_etm/@$1/u --per-thread \
> +		-- taskset -c $2 touch $file
> +}
> +
> +perf_script_branch_samples() {
> +	echo "Looking at perf.data file for dumping branch samples:"
> +
> +	# Below is an example of the branch samples dumping:
> +	#   touch  6512          1         branches:u:      ffffb220824c strcmp+0xc (/lib/aarch64-linux-gnu/ld-2.27.so)
> +	#   touch  6512          1         branches:u:      ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
> +	#   touch  6512          1         branches:u:      ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
> +	perf script -F,-time -i ${perfdata} | \
> +		egrep " +$1 +[0-9]+ .* +branches:([u|k]:)? +"
> +}
> +
> +perf_report_branch_samples() {
> +	echo "Looking at perf.data file for reporting branch samples:"
> +
> +	# Below is an example of the branch samples reporting:
> +	#   73.04%    73.04%  touch    libc-2.27.so      [.] _dl_addr
> +	#    7.71%     7.71%  touch    libc-2.27.so      [.] getenv
> +	#    2.59%     2.59%  touch    ld-2.27.so        [.] strcmp
> +	perf report --stdio -i ${perfdata} | \
> +		egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 "
> +}
> +
> +perf_report_instruction_samples() {
> +	echo "Looking at perf.data file for instruction samples:"
> +
> +	# Below is an example of the instruction samples reporting:
> +	#   68.12%  touch    libc-2.27.so   [.] _dl_addr
> +	#    5.80%  touch    libc-2.27.so   [.] getenv
> +	#    4.35%  touch    ld-2.27.so     [.] _dl_fixup
> +	perf report --itrace=i1000i --stdio -i ${perfdata} | \
> +		egrep " +[0-9]+\.[0-9]+% +$1"
> +}
> +
> +arm_cs_iterate_devices() {
> +	for dev in $1/connections/out\:*; do
> +
> +		# Skip testing if it's not a directory
> +		! [ -d $dev ] && continue;
> +
> +		# Read out its symbol link file name
> +		path=`readlink -f $dev`
> +
> +		# Extract device name from path, e.g.
> +		#   path = '/sys/devices/platform/20010000.etf/tmc_etf0'
> +		#     `> device_name = 'tmc_etf0'
> +		device_name=`echo $path | awk -F/ '{print $(NF)}'`
> +
> +		echo $device_name | egrep -q "etr|etb|etf"

Could we check for the existence of "enable_sink" instead, for detecting
if this is a sink device ? That way, we are covered for future cases of
a new sink type, and is more reliable.


> +
> +		# Only test if the output device is ETR/ETB/ETF
> +		if [ $? -eq 0 ]; then
> +
> +			pmu_dev="/sys/bus/event_source/devices/cs_etm/sinks/$device_name"
> +
> +			# Exit if PMU device node doesn't exist
> +			if ! [ -f $pmu_dev ]; then
> +				echo "PMU device $pmu_dev doesn't exist"

Misleading output. $pmu_dev is not a PMU device. Instead, it is one of
the supported sinks by the PMU.

> +				exit 1
> +			fi
> +
> +			record_touch_file $device_name $2 &&
> +				perf_script_branch_samples touch &&
> +				perf_report_branch_samples touch &&
> +				perf_report_instruction_samples touch
> +
> +			err=$?
> +
> +			# Exit when find failure
> +			[ $err != 0 ] && exit $err
> +
> +			rm -f ${perfdata}
> +			rm -f ${file}
> +		fi
> +
> +		arm_cs_iterate_devices $dev $2
> +	done
> +}
> +
> +arm_cs_etm_traverse_path_test() {
> +	# Iterate for every ETM device
> +	for dev in /sys/bus/coresight/devices/etm*; do
> +
> +		# Find the ETM device belonging to which CPU
> +		cpu=`cat $dev/cpu`
> +
> +		echo $dev
> +		echo $cpu
> +
> +		# Use depth-first search (DFS) to iterate outputs
> +		arm_cs_iterate_devices $dev $cpu
> +	done
> +}
> +
> +arm_cs_etm_cpu_wide_test() {
> +	echo "Recording trace with CPU wide mode"
> +	perf record -o ${perfdata} -e cs_etm// -a -- ls
> +
> +	perf_script_branch_samples perf &&
> +	perf_report_branch_samples perf &&
> +	perf_report_instruction_samples perf
> +
> +	err=$?
> +
> +	# Exit when find failure
> +	[ $err != 0 ] && exit $err
> +
> +	rm -f ${perfdata}
> +	rm -f ${file}
> +}
> +
> +arm_cs_etm_snapshot_test() {
> +	echo "Recording trace with snapshot mode"
> +	perf record -o ${perfdata} -e cs_etm// -S --per-thread \
> +		-- dd if=/dev/zero of=/dev/null &

As far as I understand, --per-thread option is not needed anymore
for normal tracing (irrespective of whether your application is
multi-threaded or not)

> +	PERFPID=$!
> +
> +	# Wait for perf program
> +	sleep 1
> +
> +	# Send signal to snapshot trace data
> +	kill -USR2 $PERFPID
> +
> +	# Stop perf program
> +	kill $PERFPID
> +	wait $PERFPID
> +
> +	perf_script_branch_samples dd &&
> +	perf_report_branch_samples dd &&
> +	perf_report_instruction_samples dd
> +
> +	err=$?
> +
> +	# Exit when find failure
> +	[ $err != 0 ] && exit $err
> +
> +	rm -f ${perfdata}
> +	rm -f ${file}
> +}
> +
> +arm_cs_etm_traverse_path_test
> +arm_cs_etm_cpu_wide_test
> +arm_cs_etm_snapshot_test
> +exit 0
> 


Rest looks OK to me.

Cheers
Suzuki

Re: [PATCH v4] perf test: Introduce script for Arm CoreSight testing

[Leo Yan]

Hi Suzuki,

On Wed, Aug 12, 2020 at 05:59:55PM +0100, Suzuki Kuruppassery Poulose wrote:
> Hi Leo,
> 
> On 08/06/2020 08:02 AM, Leo Yan wrote:
> > We need a simple method to test Perf with Arm CoreSight drivers, this
> > could be used for smoke testing when new patch is coming for perf or
> > CoreSight drivers, and we also can use the test to confirm if the
> > CoreSight has been enabled successfully on new platforms.
> > 
> > This patch introduces the shell script test_arm_coresight.sh which is
> > under the 'pert test' framework.  This script provides three testing
> > scenarios:
> 
> Thank you for this testcase. It is a very good tool for people
> check their system for CoreSight driver functionality.

My pleasure!  This patch is except for ourselves testing, one main
purpose is to integrate Perf testing with LKFT [1] for Arm related
PMU events.

[1] https://lkft.linaro.org/tests/

> > Test scenario 1: traverse all possible paths between source and sink
> > 
> > For traversing possible paths, simply to say, the testing rationale
> > is source oriented testing, it traverses every source (now only refers
> > to ETM device) and test its all possible sinks.  To search the complete
> > paths from one specific source to its sinks, this patch relies on the
> > sysfs '/sys/bus/coresight/devices/devX/out:Y' for depth-first search
> > (DFS) for iteration connected device nodes, if the output device is
> > detected as one of ETR, ETF, or ETB types then it will test trace data
> 
> Please see my suggestion below, to use "enable_sink" as an indicator
> for a sink device.
> 
> > recording and decoding for this PMU device.
> > 
> > The script runs three output testings for every trace data:
> > - Test branch samples dumping with 'perf script' command;
> > - Test branch samples reporting with 'perf report' command;
> > - Use option '--itrace=i1000i' to insert synthesized instructions events
> >    and the script will check if perf can output the percentage value
> >    successfully based on the instruction samples.
> > 
> > Test scenario 2: CPU wide mode test
> >  > For CPU wide mode testing, it passes option '-a' to perf tool to enable
> > tracing on all CPUs, so it's hard to say which program will be traced.
> 
> Isn't this system-wide, when you trace all CPUs ? In CPU wide mode,
> you specify a list of CPUs (-C ?). I always get confused here.

Agree, it is system-wide tracing.  Will fix.

> > But perf tool itself contributes much overload in this case, so it will
> > parse trace data and check if process 'perf' can be detected or not.
> > 
> > diff --git a/tools/perf/tests/shell/test_arm_coresight.sh b/tools/perf/tests/shell/test_arm_coresight.sh
> > new file mode 100755
> > index 000000000000..73b973bada26
> > --- /dev/null
> > +++ b/tools/perf/tests/shell/test_arm_coresight.sh
> > @@ -0,0 +1,172 @@
> > +#!/bin/sh
> > +# Check Arm CoreSight trace data recording and branch samples
> > +
> > +# Uses the 'perf record' to record trace data with Arm CoreSight sinks;
> > +# then verify if there have any branch samples and instruction samples
> > +# are generated by CoreSight with 'perf script' and 'perf report'
> > +# commands.
> > +
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Leo Yan <leo.yan@linaro.org>, 2020
> > +
> > +perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
> > +file=$(mktemp /tmp/temporary_file.XXXXX)
> > +
> > +skip_if_no_cs_etm_event() {
> > +	perf list | grep -q 'cs_etm//' && return 0
> > +
> > +	# cs_etm event doesn't exist
> > +	return 2
> > +}
> > +
> > +skip_if_no_cs_etm_event || exit 2
> > +
> > +record_touch_file() {
> > +	echo "Recording trace (only user mode) with path: CPU$2 => $1"
> > +	perf record -o ${perfdata} -e cs_etm/@$1/u --per-thread \
> > +		-- taskset -c $2 touch $file
> > +}
> > +
> > +perf_script_branch_samples() {
> > +	echo "Looking at perf.data file for dumping branch samples:"
> > +
> > +	# Below is an example of the branch samples dumping:
> > +	#   touch  6512          1         branches:u:      ffffb220824c strcmp+0xc (/lib/aarch64-linux-gnu/ld-2.27.so)
> > +	#   touch  6512          1         branches:u:      ffffb22082e0 strcmp+0xa0 (/lib/aarch64-linux-gnu/ld-2.27.so)
> > +	#   touch  6512          1         branches:u:      ffffb2208320 strcmp+0xe0 (/lib/aarch64-linux-gnu/ld-2.27.so)
> > +	perf script -F,-time -i ${perfdata} | \
> > +		egrep " +$1 +[0-9]+ .* +branches:([u|k]:)? +"
> > +}
> > +
> > +perf_report_branch_samples() {
> > +	echo "Looking at perf.data file for reporting branch samples:"
> > +
> > +	# Below is an example of the branch samples reporting:
> > +	#   73.04%    73.04%  touch    libc-2.27.so      [.] _dl_addr
> > +	#    7.71%     7.71%  touch    libc-2.27.so      [.] getenv
> > +	#    2.59%     2.59%  touch    ld-2.27.so        [.] strcmp
> > +	perf report --stdio -i ${perfdata} | \
> > +		egrep " +[0-9]+\.[0-9]+% +[0-9]+\.[0-9]+% +$1 "
> > +}
> > +
> > +perf_report_instruction_samples() {
> > +	echo "Looking at perf.data file for instruction samples:"
> > +
> > +	# Below is an example of the instruction samples reporting:
> > +	#   68.12%  touch    libc-2.27.so   [.] _dl_addr
> > +	#    5.80%  touch    libc-2.27.so   [.] getenv
> > +	#    4.35%  touch    ld-2.27.so     [.] _dl_fixup
> > +	perf report --itrace=i1000i --stdio -i ${perfdata} | \
> > +		egrep " +[0-9]+\.[0-9]+% +$1"
> > +}
> > +
> > +arm_cs_iterate_devices() {
> > +	for dev in $1/connections/out\:*; do
> > +
> > +		# Skip testing if it's not a directory
> > +		! [ -d $dev ] && continue;
> > +
> > +		# Read out its symbol link file name
> > +		path=`readlink -f $dev`
> > +
> > +		# Extract device name from path, e.g.
> > +		#   path = '/sys/devices/platform/20010000.etf/tmc_etf0'
> > +		#     `> device_name = 'tmc_etf0'
> > +		device_name=`echo $path | awk -F/ '{print $(NF)}'`
> > +
> > +		echo $device_name | egrep -q "etr|etb|etf"
> 
> Could we check for the existence of "enable_sink" instead, for detecting
> if this is a sink device ? That way, we are covered for future cases of
> a new sink type, and is more reliable.

Good suggestion, will fix.

> > +
> > +		# Only test if the output device is ETR/ETB/ETF
> > +		if [ $? -eq 0 ]; then
> > +
> > +			pmu_dev="/sys/bus/event_source/devices/cs_etm/sinks/$device_name"
> > +
> > +			# Exit if PMU device node doesn't exist
> > +			if ! [ -f $pmu_dev ]; then
> > +				echo "PMU device $pmu_dev doesn't exist"
> 
> Misleading output. $pmu_dev is not a PMU device. Instead, it is one of
> the supported sinks by the PMU.

Will fix.

> > +				exit 1
> > +			fi
> > +
> > +			record_touch_file $device_name $2 &&
> > +				perf_script_branch_samples touch &&
> > +				perf_report_branch_samples touch &&
> > +				perf_report_instruction_samples touch
> > +
> > +			err=$?
> > +
> > +			# Exit when find failure
> > +			[ $err != 0 ] && exit $err
> > +
> > +			rm -f ${perfdata}
> > +			rm -f ${file}
> > +		fi
> > +
> > +		arm_cs_iterate_devices $dev $2
> > +	done
> > +}
> > +
> > +arm_cs_etm_traverse_path_test() {
> > +	# Iterate for every ETM device
> > +	for dev in /sys/bus/coresight/devices/etm*; do
> > +
> > +		# Find the ETM device belonging to which CPU
> > +		cpu=`cat $dev/cpu`
> > +
> > +		echo $dev
> > +		echo $cpu
> > +
> > +		# Use depth-first search (DFS) to iterate outputs
> > +		arm_cs_iterate_devices $dev $cpu
> > +	done
> > +}
> > +
> > +arm_cs_etm_cpu_wide_test() {
> > +	echo "Recording trace with CPU wide mode"
> > +	perf record -o ${perfdata} -e cs_etm// -a -- ls
> > +
> > +	perf_script_branch_samples perf &&
> > +	perf_report_branch_samples perf &&
> > +	perf_report_instruction_samples perf
> > +
> > +	err=$?
> > +
> > +	# Exit when find failure
> > +	[ $err != 0 ] && exit $err
> > +
> > +	rm -f ${perfdata}
> > +	rm -f ${file}
> > +}
> > +
> > +arm_cs_etm_snapshot_test() {
> > +	echo "Recording trace with snapshot mode"
> > +	perf record -o ${perfdata} -e cs_etm// -S --per-thread \
> > +		-- dd if=/dev/zero of=/dev/null &
> 
> As far as I understand, --per-thread option is not needed anymore
> for normal tracing (irrespective of whether your application is
> multi-threaded or not)

I noted before you had some discussion with Mathieu for AUX buffer :)
will remove --per-thread option.

> > +	PERFPID=$!
> > +
> > +	# Wait for perf program
> > +	sleep 1
> > +
> > +	# Send signal to snapshot trace data
> > +	kill -USR2 $PERFPID
> > +
> > +	# Stop perf program
> > +	kill $PERFPID
> > +	wait $PERFPID
> > +
> > +	perf_script_branch_samples dd &&
> > +	perf_report_branch_samples dd &&
> > +	perf_report_instruction_samples dd
> > +
> > +	err=$?
> > +
> > +	# Exit when find failure
> > +	[ $err != 0 ] && exit $err
> > +
> > +	rm -f ${perfdata}
> > +	rm -f ${file}
> > +}
> > +
> > +arm_cs_etm_traverse_path_test
> > +arm_cs_etm_cpu_wide_test
> > +arm_cs_etm_snapshot_test
> > +exit 0
> > 
> 
> 
> Rest looks OK to me.

Thanks a lot for your reviewing.

Leo