* [RFC PATCH 0/4] Add VG register attr test with kernel version and feature detection
@ 2022-09-27 15:41 James Clark
2022-09-27 15:41 ` [RFC PATCH 1/4] perf test: Add ability to test exit code for attr tests James Clark
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: James Clark @ 2022-09-27 15:41 UTC (permalink / raw)
To: acme, namhyung
Cc: linux-kernel, linux-perf-users, will, James Clark,
Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
Jiri Olsa, bpf
This is the kind of test that I would like to add for VG, with the
necessary additions to the attr test framework for kernel version
detection so that it always passes even when run on old kernels.
This approach isn't very scalable, for example the change to fix the
tests for PERF_FORMAT_LOST [1] affects many tests so they would all have
to be duplicated. Of course there is still the option to weaken those
tests and make them accept any value.
It also doesn't apply to any of the shell script tests which would need
to re-implement something similar to this.
I'm interested to see if people think this is useful, at least it is for
this one test.
[1]: https://lore.kernel.org/linux-perf-users/20220926144948.223641-1-james.clark@arm.com/T/#t
James Clark (4):
perf test: Add ability to test exit code for attr tests
perf test: Add mechanism for skipping attr tests on auxiliary vector
values
perf test: Add mechanism for skipping attr tests on kernel versions
perf test arm64: Add attr tests for new VG register
tools/perf/tests/attr.py | 71 +++++++++++++++++--
.../attr/test-record-user-regs-no-sve-aarch64 | 9 +++
.../test-record-user-regs-old-sve-aarch64 | 10 +++
.../attr/test-record-user-regs-sve-aarch64 | 14 ++++
4 files changed, 99 insertions(+), 5 deletions(-)
create mode 100644 tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64
create mode 100644 tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64
create mode 100644 tools/perf/tests/attr/test-record-user-regs-sve-aarch64
--
2.28.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [RFC PATCH 1/4] perf test: Add ability to test exit code for attr tests
2022-09-27 15:41 [RFC PATCH 0/4] Add VG register attr test with kernel version and feature detection James Clark
@ 2022-09-27 15:41 ` James Clark
2022-09-27 15:41 ` [RFC PATCH 2/4] perf test: Add mechanism for skipping attr tests on auxiliary vector values James Clark
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: James Clark @ 2022-09-27 15:41 UTC (permalink / raw)
To: acme, namhyung
Cc: linux-kernel, linux-perf-users, will, James Clark,
Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
Jiri Olsa, bpf
Currently the return value is used to skip the test, but sometimes it
can be useful to test if a certain command should return a certain exit
code.
Signed-off-by: James Clark <james.clark@arm.com>
---
tools/perf/tests/attr.py | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py
index 6d9b5312e6c2..7e053fa5d6ae 100644
--- a/tools/perf/tests/attr.py
+++ b/tools/perf/tests/attr.py
@@ -129,7 +129,8 @@ class Event(dict):
# - needs to specify:
# 'command' - perf command name
# 'args' - special command arguments
-# 'ret' - expected command return value (0 by default)
+# 'ret' - Skip test if Perf doesn't exit with this value (0 by default)
+# 'test_ret'- If set to 'true', fail test instead of skipping for 'ret' argument
# 'arch' - architecture specific test (optional)
# comma separated list, ! at the beginning
# negates it.
@@ -155,6 +156,8 @@ class Test(object):
except:
self.ret = 0
+ self.test_ret = parser.getboolean('config', 'test_ret', fallback=False)
+
try:
self.arch = parser.get('config', 'arch')
log.warning("test limitation '%s'" % self.arch)
@@ -232,7 +235,10 @@ class Test(object):
log.info(" '%s' ret '%s', expected '%s'" % (cmd, str(ret), str(self.ret)))
if not data_equal(str(ret), str(self.ret)):
- raise Unsup(self)
+ if self.test_ret:
+ raise Fail(self, "Perf exit code failure")
+ else:
+ raise Unsup(self)
def compare(self, expect, result):
match = {}
--
2.28.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [RFC PATCH 2/4] perf test: Add mechanism for skipping attr tests on auxiliary vector values
2022-09-27 15:41 [RFC PATCH 0/4] Add VG register attr test with kernel version and feature detection James Clark
2022-09-27 15:41 ` [RFC PATCH 1/4] perf test: Add ability to test exit code for attr tests James Clark
@ 2022-09-27 15:41 ` James Clark
2022-09-27 15:41 ` [RFC PATCH 3/4] perf test: Add mechanism for skipping attr tests on kernel versions James Clark
2022-09-27 15:41 ` [RFC PATCH 4/4] perf test arm64: Add attr tests for new VG register James Clark
3 siblings, 0 replies; 5+ messages in thread
From: James Clark @ 2022-09-27 15:41 UTC (permalink / raw)
To: acme, namhyung
Cc: linux-kernel, linux-perf-users, will, James Clark,
Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
Jiri Olsa, bpf
This can be used to skip tests or provide different test values on
different platforms. For example to run a test only where Arm SVE is
present add this to the config section:
auxv = auxv["AT_HWCAP"] & 0x200000 == 0x200000
The value is a freeform Python expression that is evaled in the context
of a map called "auxv" that contains the decoded auxiliary vector.
Signed-off-by: James Clark <james.clark@arm.com>
---
tools/perf/tests/attr.py | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py
index 7e053fa5d6ae..2d6124801da0 100644
--- a/tools/perf/tests/attr.py
+++ b/tools/perf/tests/attr.py
@@ -8,7 +8,9 @@ import glob
import optparse
import tempfile
import logging
+import re
import shutil
+import subprocess
try:
import configparser
@@ -134,6 +136,8 @@ class Event(dict):
# 'arch' - architecture specific test (optional)
# comma separated list, ! at the beginning
# negates it.
+# 'auxv' - Truthy statement that is evaled in the scope of the auxv map. When false,
+# the test is skipped. For example 'auxv["AT_HWCAP"] == 10'. (optional)
#
# [eventX:base]
# - one or multiple instances in file
@@ -164,6 +168,7 @@ class Test(object):
except:
self.arch = ''
+ self.auxv = parser.get('config', 'auxv', fallback=None)
self.expect = {}
self.result = {}
log.debug(" loading expected events");
@@ -175,7 +180,28 @@ class Test(object):
else:
return True
- def skip_test(self, myarch):
+ def skip_test_auxv(self):
+ def new_auxv(a, pattern):
+ items = list(filter(None, pattern.split(a)))
+ # AT_HWCAP is hex but doesn't have a prefix, so special case it
+ if items[0] == "AT_HWCAP":
+ value = int(items[-1], 16)
+ else:
+ try:
+ value = int(items[-1], 0)
+ except:
+ value = items[-1]
+ return (items[0], value)
+
+ if not self.auxv:
+ return False
+ auxv = subprocess.check_output("LD_SHOW_AUXV=1 sleep 0", shell=True) \
+ .decode(sys.stdout.encoding)
+ pattern = re.compile(r"[: ]+")
+ auxv = dict([new_auxv(a, pattern) for a in auxv.splitlines()])
+ return not eval(self.auxv)
+
+ def skip_test_arch(self, myarch):
# If architecture not set always run test
if self.arch == '':
# log.warning("test for arch %s is ok" % myarch)
@@ -225,9 +251,12 @@ class Test(object):
def run_cmd(self, tempdir):
junk1, junk2, junk3, junk4, myarch = (os.uname())
- if self.skip_test(myarch):
+ if self.skip_test_arch(myarch):
raise Notest(self, myarch)
+ if self.skip_test_auxv():
+ raise Notest(self, "auxv skip")
+
cmd = "PERF_TEST_ATTR=%s %s %s -o %s/perf.data %s" % (tempdir,
self.perf, self.command, tempdir, self.args)
ret = os.WEXITSTATUS(os.system(cmd))
--
2.28.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [RFC PATCH 3/4] perf test: Add mechanism for skipping attr tests on kernel versions
2022-09-27 15:41 [RFC PATCH 0/4] Add VG register attr test with kernel version and feature detection James Clark
2022-09-27 15:41 ` [RFC PATCH 1/4] perf test: Add ability to test exit code for attr tests James Clark
2022-09-27 15:41 ` [RFC PATCH 2/4] perf test: Add mechanism for skipping attr tests on auxiliary vector values James Clark
@ 2022-09-27 15:41 ` James Clark
2022-09-27 15:41 ` [RFC PATCH 4/4] perf test arm64: Add attr tests for new VG register James Clark
3 siblings, 0 replies; 5+ messages in thread
From: James Clark @ 2022-09-27 15:41 UTC (permalink / raw)
To: acme, namhyung
Cc: linux-kernel, linux-perf-users, will, James Clark,
Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
Jiri Olsa, bpf
The first two version numbers are used since that is where the ABI
changes happen, so seems to be the most useful for now.
'Until' is exclusive and 'since' is inclusive so that the same version
number can be used to mark a point where the change comes into effect.
This allows keeping the tests in a state where new tests will also pass
on older kernels if the existence of a new feature isn't explicitly
broadcast by the kernel. For example extended user regs are currently
discovered by trial and error calls to perf_event_open.
Signed-off-by: James Clark <james.clark@arm.com>
---
tools/perf/tests/attr.py | 28 +++++++++++++++++++++++++++-
1 file changed, 27 insertions(+), 1 deletion(-)
diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py
index 2d6124801da0..e890c261ad26 100644
--- a/tools/perf/tests/attr.py
+++ b/tools/perf/tests/attr.py
@@ -6,6 +6,7 @@ import os
import sys
import glob
import optparse
+import platform
import tempfile
import logging
import re
@@ -125,6 +126,11 @@ class Event(dict):
if not data_equal(self[t], other[t]):
log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
+def parse_version(version):
+ if not version:
+ return None
+ return [int(v) for v in version.split(".")[0:2]]
+
# Test file description needs to have following sections:
# [config]
# - just single instance in file
@@ -138,7 +144,9 @@ class Event(dict):
# negates it.
# 'auxv' - Truthy statement that is evaled in the scope of the auxv map. When false,
# the test is skipped. For example 'auxv["AT_HWCAP"] == 10'. (optional)
-#
+# 'kernel_since' - Inclusive kernel version from which the test will start running. Only the
+# first two values are supported, for example "6.1" (optional)
+# 'kernel_until' - Exclusive kernel version from which the test will stop running. (optional)
# [eventX:base]
# - one or multiple instances in file
# - expected values assignments
@@ -169,6 +177,8 @@ class Test(object):
self.arch = ''
self.auxv = parser.get('config', 'auxv', fallback=None)
+ self.kernel_since = parse_version(parser.get('config', 'kernel_since', fallback=None))
+ self.kernel_until = parse_version(parser.get('config', 'kernel_until', fallback=None))
self.expect = {}
self.result = {}
log.debug(" loading expected events");
@@ -180,6 +190,16 @@ class Test(object):
else:
return True
+ def skip_test_kernel_since(self):
+ if not self.kernel_since:
+ return False
+ return not self.kernel_since <= parse_version(platform.release())
+
+ def skip_test_kernel_until(self):
+ if not self.kernel_until:
+ return False
+ return not parse_version(platform.release()) < self.kernel_until
+
def skip_test_auxv(self):
def new_auxv(a, pattern):
items = list(filter(None, pattern.split(a)))
@@ -257,6 +277,12 @@ class Test(object):
if self.skip_test_auxv():
raise Notest(self, "auxv skip")
+ if self.skip_test_kernel_since():
+ raise Notest(self, "old kernel skip")
+
+ if self.skip_test_kernel_until():
+ raise Notest(self, "new kernel skip")
+
cmd = "PERF_TEST_ATTR=%s %s %s -o %s/perf.data %s" % (tempdir,
self.perf, self.command, tempdir, self.args)
ret = os.WEXITSTATUS(os.system(cmd))
--
2.28.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [RFC PATCH 4/4] perf test arm64: Add attr tests for new VG register
2022-09-27 15:41 [RFC PATCH 0/4] Add VG register attr test with kernel version and feature detection James Clark
` (2 preceding siblings ...)
2022-09-27 15:41 ` [RFC PATCH 3/4] perf test: Add mechanism for skipping attr tests on kernel versions James Clark
@ 2022-09-27 15:41 ` James Clark
3 siblings, 0 replies; 5+ messages in thread
From: James Clark @ 2022-09-27 15:41 UTC (permalink / raw)
To: acme, namhyung
Cc: linux-kernel, linux-perf-users, will, James Clark,
Peter Zijlstra, Ingo Molnar, Mark Rutland, Alexander Shishkin,
Jiri Olsa, bpf
Ensure that the availability of the VG register behaves as expected
depending on the kernel version and SVE support.
Signed-off-by: James Clark <james.clark@arm.com>
---
.../attr/test-record-user-regs-no-sve-aarch64 | 9 +++++++++
.../attr/test-record-user-regs-old-sve-aarch64 | 10 ++++++++++
.../tests/attr/test-record-user-regs-sve-aarch64 | 14 ++++++++++++++
3 files changed, 33 insertions(+)
create mode 100644 tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64
create mode 100644 tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64
create mode 100644 tools/perf/tests/attr/test-record-user-regs-sve-aarch64
diff --git a/tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64 b/tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64
new file mode 100644
index 000000000000..fbb065842880
--- /dev/null
+++ b/tools/perf/tests/attr/test-record-user-regs-no-sve-aarch64
@@ -0,0 +1,9 @@
+# Test that asking for VG fails if the system doesn't support SVE. This
+# applies both before and after the feature was added in 6.1
+[config]
+command = record
+args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1
+ret = 129
+test_ret = true
+arch = aarch64
+auxv = auxv["AT_HWCAP"] & 0x200000 == 0
diff --git a/tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64 b/tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64
new file mode 100644
index 000000000000..15ebfc3418e3
--- /dev/null
+++ b/tools/perf/tests/attr/test-record-user-regs-old-sve-aarch64
@@ -0,0 +1,10 @@
+# Test that asking for VG always fails on old kernels because it was
+# added in 6.1. This applies to systems that either support or don't
+# support SVE.
+[config]
+command = record
+args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1
+ret = 129
+test_ret = true
+arch = aarch64
+kernel_until = 6.1
diff --git a/tools/perf/tests/attr/test-record-user-regs-sve-aarch64 b/tools/perf/tests/attr/test-record-user-regs-sve-aarch64
new file mode 100644
index 000000000000..c598c803221d
--- /dev/null
+++ b/tools/perf/tests/attr/test-record-user-regs-sve-aarch64
@@ -0,0 +1,14 @@
+# Test that asking for VG works if the system has SVE and after the
+# feature was added in 6.1
+[config]
+command = record
+args = --no-bpf-event --user-regs=vg kill >/dev/null 2>&1
+ret = 1
+test_ret = true
+arch = aarch64
+auxv = auxv["AT_HWCAP"] & 0x200000 == 0x200000
+kernel_since = 6.1
+
+[event:base-record]
+sample_type=4359
+sample_regs_user=70368744177664
--
2.28.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-09-27 15:45 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-27 15:41 [RFC PATCH 0/4] Add VG register attr test with kernel version and feature detection James Clark
2022-09-27 15:41 ` [RFC PATCH 1/4] perf test: Add ability to test exit code for attr tests James Clark
2022-09-27 15:41 ` [RFC PATCH 2/4] perf test: Add mechanism for skipping attr tests on auxiliary vector values James Clark
2022-09-27 15:41 ` [RFC PATCH 3/4] perf test: Add mechanism for skipping attr tests on kernel versions James Clark
2022-09-27 15:41 ` [RFC PATCH 4/4] perf test arm64: Add attr tests for new VG register James Clark
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.