[PATCH 0/4] perf bpf_counter: A set of random fixes (v1)

[PATCH 0/4] perf bpf_counter: A set of random fixes (v1)

[Namhyung Kim]

Hello,

This is a collection of small fixes for perf stat bpf counters (bperf).

The bperf framework maintains perf_attr_map in the BPF fs to share the
same event as much as possible.  But the size was limited to 16 and
perf stat with -ddd option would create more than 16 events and fails.

Also cgroup events with --for-each-cgroup had some other problems when
dealing with unsupported events and duplicate cgroups.

The code is available at 'perf/stat-bpf-fix-v1' branch in

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung

Namhyung Kim (4):
  perf bpf_counter: Add more error messages for bperf
  perf bpf_counter: Increase perf_attr_map entries to 32
  perf bpf_counter: Handle unsupported cgroup events
  perf stat: Do not use the same cgroup more than once

 tools/perf/util/bpf_counter.c        | 11 ++++++++---
 tools/perf/util/bpf_counter_cgroup.c | 14 +++-----------
 tools/perf/util/cgroup.c             | 23 ++++++++++++++++++-----
 3 files changed, 29 insertions(+), 19 deletions(-)


base-commit: d8d85ce86dc82de4f88b821a78f533b9d5b22a45
-- 
2.39.0.314.g84b9a713c41-goog

[PATCH 1/4] perf bpf_counter: Add more error messages for bperf

[Namhyung Kim]

I found perf stat silently exits when it failed in bperf__load().
Let's add some error messages to notify users.

  $ sudo ./perf stat -a --bpf-counters -ddd sleep 1
  bpf_map_update_elem failed: err=-7

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/bpf_counter.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/bpf_counter.c b/tools/perf/util/bpf_counter.c
index eeee899fcf34..7f5cc1aa4903 100644
--- a/tools/perf/util/bpf_counter.c
+++ b/tools/perf/util/bpf_counter.c
@@ -471,8 +471,10 @@ static int bperf__load(struct evsel *evsel, struct target *target)
 
 	if (!all_cpu_map) {
 		all_cpu_map = perf_cpu_map__new(NULL);
-		if (!all_cpu_map)
+		if (!all_cpu_map) {
+			pr_err("failed to create all cpu map\n");
 			return -1;
+		}
 	}
 
 	evsel->bperf_leader_prog_fd = -1;
@@ -493,13 +495,16 @@ static int bperf__load(struct evsel *evsel, struct target *target)
 	err = bpf_map_lookup_elem(attr_map_fd, &evsel->core.attr, &entry);
 	if (err) {
 		err = bpf_map_update_elem(attr_map_fd, &evsel->core.attr, &entry, BPF_ANY);
-		if (err)
+		if (err) {
+			pr_err("updating perf_event_attr map failed: err=%d\n", err);
 			goto out;
+		}
 	}
 
 	evsel->bperf_leader_link_fd = bpf_link_get_fd_by_id(entry.link_id);
 	if (evsel->bperf_leader_link_fd < 0 &&
 	    bperf_reload_leader_program(evsel, attr_map_fd, &entry)) {
+		pr_err("reload leader program failed\n");
 		err = -1;
 		goto out;
 	}
-- 
2.39.0.314.g84b9a713c41-goog

[PATCH 2/4] perf bpf_counter: Increase perf_attr_map entries to 32

[Namhyung Kim]

The current size 16 cannot hold all events when user gave -dd or -ddd.
As it's a part of perf stat, let's increase the size to 32.

  # unlink previous map to change the size
  $ sudo unlink /sys/fs/bpf/perf_attr_map

  $ sudo ./perf stat -a --bpf-counters -ddd sleep 1

   Performance counter stats for 'system wide':

           35,927.41 msec cpu-clock                        #   35.802 CPUs utilized
              12,629      context-switches                 #  351.514 /sec
                 209      cpu-migrations                   #    5.817 /sec
                 826      page-faults                      #   22.991 /sec
       2,155,729,621      cycles                           #    0.060 GHz                         (30.43%)
       1,053,849,500      instructions                     #    0.49  insn per cycle              (38.18%)
         232,711,500      branches                         #    6.477 M/sec                       (38.44%)
          10,693,352      branch-misses                    #    4.60% of all branches             (31.27%)
         267,561,655      L1-dcache-loads                  #    7.447 M/sec                       (30.58%)
          27,290,728      L1-dcache-load-misses            #   10.20% of all L1-dcache accesses   (30.48%)
          12,651,208      LLC-loads                        #  352.133 K/sec                       (30.78%)
           1,274,018      LLC-load-misses                  #   10.07% of all LL-cache accesses    (38.70%)
     <not supported>      L1-icache-loads
          75,916,358      L1-icache-load-misses            #    0.00% of all L1-icache accesses   (38.11%)
         273,330,559      dTLB-loads                       #    7.608 M/sec                       (30.54%)
           2,864,458      dTLB-load-misses                 #    1.05% of all dTLB cache accesses  (38.77%)
             361,507      iTLB-loads                       #   10.062 K/sec                       (30.59%)
             848,031      iTLB-load-misses                 #  234.58% of all iTLB cache accesses  (31.30%)
     <not supported>      L1-dcache-prefetches
     <not supported>      L1-dcache-prefetch-misses

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/bpf_counter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/bpf_counter.c b/tools/perf/util/bpf_counter.c
index 7f5cc1aa4903..de6331f5263c 100644
--- a/tools/perf/util/bpf_counter.c
+++ b/tools/perf/util/bpf_counter.c
@@ -28,7 +28,7 @@
 #include "bpf_skel/bperf_leader.skel.h"
 #include "bpf_skel/bperf_follower.skel.h"
 
-#define ATTR_MAP_SIZE 16
+#define ATTR_MAP_SIZE 32
 
 static inline void *u64_to_ptr(__u64 ptr)
 {
-- 
2.39.0.314.g84b9a713c41-goog

[PATCH 3/4] perf bpf_counter: Handle unsupported cgroup events

[Namhyung Kim]

When --for-each-cgroup option is used, it failed when any of events is not
supported and it exited immediately.  This is not how perf stat handles the
unsupported events.  Let's ignore the failure and proceed with others.

Before:
  $ sudo ./perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads \
  > --for-each-cgroup system.slice,user.slice  sleep 1
  Failed to open first cgroup events

After:
  $ sudo ./perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads \
  > --for-each-cgroup system.slice,user.slice  sleep 1

   Performance counter stats for 'system wide':

     <not supported>      L1-icache-loads                  system.slice
          29,892,418      L1-dcache-loads                  system.slice
     <not supported>      L1-icache-loads                  user.slice
          52,497,220      L1-dcache-loads                  user.slice

Fixes: 944138f048f7d ("perf stat: Enable BPF counter with --for-each-cgroup")
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/bpf_counter_cgroup.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/tools/perf/util/bpf_counter_cgroup.c b/tools/perf/util/bpf_counter_cgroup.c
index 3c2df7522f6f..1c82377ed78b 100644
--- a/tools/perf/util/bpf_counter_cgroup.c
+++ b/tools/perf/util/bpf_counter_cgroup.c
@@ -116,27 +116,19 @@ static int bperf_load_program(struct evlist *evlist)
 
 			/* open single copy of the events w/o cgroup */
 			err = evsel__open_per_cpu(evsel, evsel->core.cpus, -1);
-			if (err) {
-				pr_err("Failed to open first cgroup events\n");
-				goto out;
-			}
+			if (err == 0)
+				evsel->supported = true;
 
 			map_fd = bpf_map__fd(skel->maps.events);
 			perf_cpu_map__for_each_cpu(cpu, j, evsel->core.cpus) {
 				int fd = FD(evsel, j);
 				__u32 idx = evsel->core.idx * total_cpus + cpu.cpu;
 
-				err = bpf_map_update_elem(map_fd, &idx, &fd,
-							  BPF_ANY);
-				if (err < 0) {
-					pr_err("Failed to update perf_event fd\n");
-					goto out;
-				}
+				bpf_map_update_elem(map_fd, &idx, &fd, BPF_ANY);
 			}
 
 			evsel->cgrp = leader_cgrp;
 		}
-		evsel->supported = true;
 
 		if (evsel->cgrp == cgrp)
 			continue;
-- 
2.39.0.314.g84b9a713c41-goog

[PATCH 4/4] perf stat: Do not use the same cgroup more than once

[Namhyung Kim]

The --for-each-cgroup can have the same cgroup multiple times, but it makes
bpf counters confused (since they have the same cgroup id), and the last
cgroup events are counted only.  Let's check the cgroup name before adding
a new entry.

Before:
  $ sudo ./perf stat -a --bpf-counters --for-each-cgroup /,/ sleep 1

   Performance counter stats for 'system wide':

       <not counted> msec cpu-clock                        /
       <not counted>      context-switches                 /
       <not counted>      cpu-migrations                   /
       <not counted>      page-faults                      /
       <not counted>      cycles                           /
       <not counted>      instructions                     /
       <not counted>      branches                         /
       <not counted>      branch-misses                    /
            8,016.04 msec cpu-clock                        /                #    7.998 CPUs utilized
               6,152      context-switches                 /                #  767.461 /sec
                 250      cpu-migrations                   /                #   31.187 /sec
                 442      page-faults                      /                #   55.139 /sec
         613,111,487      cycles                           /                #    0.076 GHz
         280,599,604      instructions                     /                #    0.46  insn per cycle
          57,692,724      branches                         /                #    7.197 M/sec
           3,385,168      branch-misses                    /                #    5.87% of all branches

         1.002220125 seconds time elapsed

After:
  $ sudo ./perf stat -a --bpf-counters --for-each-cgroup /,/  sleep 1

   Performance counter stats for 'system wide':

            8,013.38 msec cpu-clock                        /                #    7.998 CPUs utilized
               6,859      context-switches                 /                #  855.944 /sec
                 334      cpu-migrations                   /                #   41.680 /sec
                 345      page-faults                      /                #   43.053 /sec
         782,326,119      cycles                           /                #    0.098 GHz
         471,645,724      instructions                     /                #    0.60  insn per cycle
          94,963,430      branches                         /                #   11.851 M/sec
           3,685,511      branch-misses                    /                #    3.88% of all branches

         1.001864539 seconds time elapsed

Fixes: bb1c15b60b981 ("perf stat: Support regex pattern in --for-each-cgroup")
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/cgroup.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/tools/perf/util/cgroup.c b/tools/perf/util/cgroup.c
index e99b41f9be45..cd978c240e0d 100644
--- a/tools/perf/util/cgroup.c
+++ b/tools/perf/util/cgroup.c
@@ -224,6 +224,19 @@ static int add_cgroup_name(const char *fpath, const struct stat *sb __maybe_unus
 	return 0;
 }
 
+static int check_and_add_cgroup_name(const char *fpath)
+{
+	struct cgroup_name *cn;
+
+	list_for_each_entry(cn, &cgroup_list, list) {
+		if (!strcmp(cn->name, fpath))
+			return 0;
+	}
+
+	/* pretend if it's added by ftw() */
+	return add_cgroup_name(fpath, NULL, FTW_D, NULL);
+}
+
 static void release_cgroup_list(void)
 {
 	struct cgroup_name *cn;
@@ -242,7 +255,7 @@ static int list_cgroups(const char *str)
 	struct cgroup_name *cn;
 	char *s;
 
-	/* use given name as is - for testing purpose */
+	/* use given name as is when no regex is given */
 	for (;;) {
 		p = strchr(str, ',');
 		e = p ? p : eos;
@@ -253,13 +266,13 @@ static int list_cgroups(const char *str)
 			s = strndup(str, e - str);
 			if (!s)
 				return -1;
-			/* pretend if it's added by ftw() */
-			ret = add_cgroup_name(s, NULL, FTW_D, NULL);
+
+			ret = check_and_add_cgroup_name(s);
 			free(s);
-			if (ret)
+			if (ret < 0)
 				return -1;
 		} else {
-			if (add_cgroup_name("", NULL, FTW_D, NULL) < 0)
+			if (check_and_add_cgroup_name("/") < 0)
 				return -1;
 		}
 
-- 
2.39.0.314.g84b9a713c41-goog

Re: [PATCH 3/4] perf bpf_counter: Handle unsupported cgroup events

[Arnaldo Carvalho de Melo]

Em Tue, Jan 03, 2023 at 10:44:01PM -0800, Namhyung Kim escreveu:
> When --for-each-cgroup option is used, it failed when any of events is not
> supported and it exited immediately.  This is not how perf stat handles the
> unsupported events.  Let's ignore the failure and proceed with others.
> 
> Before:
>   $ sudo ./perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads \
>   > --for-each-cgroup system.slice,user.slice  sleep 1
>   Failed to open first cgroup events

Interesting, tried the above on a arm64 machine and it all works:

root@roc-rk3399-pc:~# strace -e bpf,perf_event_open perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads --for-each-cgroup system.slice,user.slice sleep 1
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0xffffd36afc98, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0}, 116) = 5
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0xffffd36afee8, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0\20\0\0\0\20\0\0\0\5\0\0\0\1\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=45, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0000\0\0\0000\0\0\0\t\0\0\0\1\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=81, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\08\0\0\08\0\0\0\t\0\0\0\0\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=89, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0\f\0\0\0\f\0\0\0\7\0\0\0\1\0\0\0\0\0\0\20"..., btf_log_buf=NULL, btf_size=43, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0000\0\0\0000\0\0\0\t\0\0\0\1\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=81, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0000\0\0\0000\0\0\0\5\0\0\0\0\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=77, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0(\0\0\0(\0\0\0\5\0\0\0\0\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=69, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0\f\0\0\0\f\0\0\0\10\0\0\0\1\0\0\0\0\0\0\23"..., btf_log_buf=NULL, btf_size=44, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\0@G\0\0@G\0\0\2725\0\0\0\0\0\0\0\0\0\2"..., btf_log_buf=NULL, btf_size=32018, btf_log_size=0, btf_log_level=0}, 28) = 5
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0xffffd36afb18, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="libbpf_nametest"}, 64) = 6
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=4, max_entries=1, map_flags=BPF_F_MMAPABLE, inner_map_fd=0, map_name="libbpf_mmap", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 6
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_PERF_EVENT_ARRAY, key_size=4, value_size=4, max_entries=12, map_flags=0, inner_map_fd=0, map_name="events", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 6
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_HASH, key_size=8, value_size=4, max_entries=2, map_flags=0, inner_map_fd=0, map_name="cgrp_idx", map_ifindex=0, btf_fd=5, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 7
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_PERCPU_ARRAY, key_size=4, value_size=24, max_entries=2, map_flags=0, inner_map_fd=0, map_name="prev_readings", map_ifindex=0, btf_fd=5, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 9
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_PERCPU_ARRAY, key_size=4, value_size=24, max_entries=4, map_flags=0, inner_map_fd=0, map_name="cgrp_readings", map_ifindex=0, btf_fd=5, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 10
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=32, max_entries=1, map_flags=0, inner_map_fd=0, map_name="libbpf_global", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 11
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=5, insns=0xffffd36afb80, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 12
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=8, max_entries=1, map_flags=BPF_F_RDONLY_PROG|BPF_F_MMAPABLE, inner_map_fd=0, map_name="bperf_cg.rodata", map_ifindex=0, btf_fd=5, btf_key_type_id=0, btf_value_type_id=413, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 11
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=11, key=0xffffd36afcdc, value=0xffff8da49000, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_FREEZE, {map_fd=11}, 4)     = 0
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=8, max_entries=1, map_flags=BPF_F_MMAPABLE, inner_map_fd=0, map_name="bperf_cg.bss", map_ifindex=0, btf_fd=5, btf_key_type_id=0, btf_value_type_id=410, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 12
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=12, key=0xffffd36afcdc, value=0xffff8da48000, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=4, max_entries=1, map_flags=BPF_F_MMAPABLE, inner_map_fd=0, map_name="bperf_cg.data", map_ifindex=0, btf_fd=5, btf_key_type_id=0, btf_value_type_id=411, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 13
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=13, key=0xffffd36afcdc, value=0xffff8da47000, flags=BPF_ANY}, 32) = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_TRACEPOINT, insn_cnt=6, insns=0xffffd36afd48, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 14
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_PERF_EVENT, insn_cnt=306, insns=0xaaaaf14b9de0, license="Dual BSD/GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(6, 1, 0), prog_flags=0, prog_name="on_cgrp_switch", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=5, func_info_rec_size=8, func_info=0xaaaaf14baec0, func_info_cnt=2, line_info_rec_size=16, line_info=0xaaaaf1491390, line_info_cnt=109, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 14
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=32, max_entries=1, map_flags=0, inner_map_fd=0, map_name="libbpf_det_bind", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0, map_extra=0}, 72) = 15
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0xffffd36af358, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 16
bpf(BPF_PROG_BIND_MAP, 0xffffd36af288, 12) = 0
bpf(BPF_PROG_BIND_MAP, 0xffffd36af398, 12) = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_RAW_TRACEPOINT, insn_cnt=306, insns=0xaaaaf1491a70, license="Dual BSD/GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(6, 1, 0), prog_flags=0, prog_name="trigger_read", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=5, func_info_rec_size=8, func_info=0xaaaaf14b9ad0, func_info_cnt=2, line_info_rec_size=16, line_info=0xaaaaf1492710, line_info_cnt=109, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 15
bpf(BPF_PROG_BIND_MAP, 0xffffd36af398, 12) = 0
perf_event_open({type=PERF_TYPE_SOFTWARE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_SW_CGROUP_SWITCHES, sample_period=1, sample_type=0, read_format=0, disabled=1, precise_ip=0 /* arbitrary skid */, ...}, -1, 0, -1, PERF_FLAG_FD_CLOEXEC) = 16
perf_event_open({type=PERF_TYPE_SOFTWARE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_SW_CGROUP_SWITCHES, sample_period=1, sample_type=0, read_format=0, disabled=1, precise_ip=0 /* arbitrary skid */, ...}, -1, 1, -1, PERF_FLAG_FD_CLOEXEC) = 17
perf_event_open({type=PERF_TYPE_SOFTWARE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_SW_CGROUP_SWITCHES, sample_period=1, sample_type=0, read_format=0, disabled=1, precise_ip=0 /* arbitrary skid */, ...}, -1, 2, -1, PERF_FLAG_FD_CLOEXEC) = 18
perf_event_open({type=PERF_TYPE_SOFTWARE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_SW_CGROUP_SWITCHES, sample_period=1, sample_type=0, read_format=0, disabled=1, precise_ip=0 /* arbitrary skid */, ...}, -1, 3, -1, PERF_FLAG_FD_CLOEXEC) = 19
perf_event_open({type=PERF_TYPE_SOFTWARE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_SW_CGROUP_SWITCHES, sample_period=1, sample_type=0, read_format=0, disabled=1, precise_ip=0 /* arbitrary skid */, ...}, -1, 4, -1, PERF_FLAG_FD_CLOEXEC) = 20
perf_event_open({type=PERF_TYPE_SOFTWARE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_SW_CGROUP_SWITCHES, sample_period=1, sample_type=0, read_format=0, disabled=1, precise_ip=0 /* arbitrary skid */, ...}, -1, 5, -1, PERF_FLAG_FD_CLOEXEC) = 21
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_TRACEPOINT, insn_cnt=2, insns=0xffffd36afe28, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(0, 0, 0), prog_flags=0, prog_name="", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=0, func_info_rec_size=0, func_info=NULL, func_info_cnt=0, line_info_rec_size=0, line_info=NULL, line_info_cnt=0, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 128) = 22
bpf(BPF_LINK_CREATE, {link_create={prog_fd=22, target_fd=-1, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = -1 EBADF (Bad file descriptor)
bpf(BPF_LINK_CREATE, {link_create={prog_fd=14, target_fd=16, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = 22
bpf(BPF_LINK_CREATE, {link_create={prog_fd=14, target_fd=17, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = 23
bpf(BPF_LINK_CREATE, {link_create={prog_fd=14, target_fd=18, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = 24
bpf(BPF_LINK_CREATE, {link_create={prog_fd=14, target_fd=19, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = 25
bpf(BPF_LINK_CREATE, {link_create={prog_fd=14, target_fd=20, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = 26
bpf(BPF_LINK_CREATE, {link_create={prog_fd=14, target_fd=21, attach_type=BPF_PERF_EVENT, flags=0}}, 48) = 27
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1I, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 0, -1, PERF_FLAG_FD_CLOEXEC) = 28
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1I, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 1, -1, PERF_FLAG_FD_CLOEXEC) = 29
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1I, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 2, -1, PERF_FLAG_FD_CLOEXEC) = 30
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1I, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 3, -1, PERF_FLAG_FD_CLOEXEC) = 31
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1I, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 4, -1, PERF_FLAG_FD_CLOEXEC) = 32
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1I, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 5, -1, PERF_FLAG_FD_CLOEXEC) = 33
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=7, key=0xaaaaf14c1c58, value=0xffffd36b005c, flags=BPF_ANY}, 32) = 0
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1D, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 0, -1, PERF_FLAG_FD_CLOEXEC) = 34
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1D, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 1, -1, PERF_FLAG_FD_CLOEXEC) = 35
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1D, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 2, -1, PERF_FLAG_FD_CLOEXEC) = 36
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1D, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 3, -1, PERF_FLAG_FD_CLOEXEC) = 37
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1D, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 4, -1, PERF_FLAG_FD_CLOEXEC) = 38
perf_event_open({type=PERF_TYPE_HW_CACHE, size=PERF_ATTR_SIZE_VER7, config=PERF_COUNT_HW_CACHE_RESULT_ACCESS<<16|PERF_COUNT_HW_CACHE_OP_READ<<8|PERF_COUNT_HW_CACHE_L1D, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, exclude_guest=1, ...}, -1, 5, -1, PERF_FLAG_FD_CLOEXEC) = 39
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=6, key=0xffffd36b00b0, value=0xffffd36b0060, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=7, key=0xaaaaf14c2278, value=0xffffd36b005c, flags=BPF_ANY}, 32) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=0}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=0}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=1}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=2}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=3}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=4}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=5}}, 80) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=8741, si_uid=0, si_status=0, si_utime=0, si_stime=1} ---
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=0}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=1}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=2}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=3}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=4}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=5}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=0}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=1}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=2}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=3}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=4}}, 80) = 0
bpf(BPF_PROG_TEST_RUN, {test={prog_fd=15, retval=0, data_size_in=0, data_size_out=0, data_in=NULL, data_out=NULL, repeat=0, duration=0, ctx_size_in=0, ctx_size_out=0, ctx_in=NULL, ctx_out=NULL, flags=BPF_F_TEST_RUN_ON_CPU, cpu=5}}, 80) = 0
bpf(BPF_MAP_LOOKUP_ELEM, {map_fd=10, key=0xffffd36b0004, value=0xaaaaf14be350, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_LOOKUP_ELEM, {map_fd=10, key=0xffffd36b0004, value=0xaaaaf14be350, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_LOOKUP_ELEM, {map_fd=10, key=0xffffd36b0004, value=0xaaaaf14be350, flags=BPF_ANY}, 32) = 0
bpf(BPF_MAP_LOOKUP_ELEM, {map_fd=10, key=0xffffd36b0004, value=0xaaaaf14be350, flags=BPF_ANY}, 32) = 0

 Performance counter stats for 'system wide':

             46116      L1-icache-loads                  system.slice
             15864      L1-dcache-loads                  system.slice
           8685400      L1-icache-loads                  user.slice
           3686787      L1-dcache-loads                  user.slice

       1.021264798 seconds time elapsed

--- SIGCHLD {si_signo=SIGCHLD, si_code=SI_USER, si_pid=8740, si_uid=0} ---
+++ exited with 0 +++
root@roc-rk3399-pc:~#
root@roc-rk3399-pc:~# perf -vv
perf version 6.2.rc2.gfb710ddee75f
                 dwarf: [ on  ]  # HAVE_DWARF_SUPPORT
    dwarf_getlocations: [ on  ]  # HAVE_DWARF_GETLOCATIONS_SUPPORT
                 glibc: [ on  ]  # HAVE_GLIBC_SUPPORT
         syscall_table: [ on  ]  # HAVE_SYSCALL_TABLE_SUPPORT
                libbfd: [ on  ]  # HAVE_LIBBFD_SUPPORT
            debuginfod: [ OFF ]  # HAVE_DEBUGINFOD_SUPPORT
                libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
               libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
               libperl: [ on  ]  # HAVE_LIBPERL_SUPPORT
             libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
              libslang: [ on  ]  # HAVE_SLANG_SUPPORT
             libcrypto: [ on  ]  # HAVE_LIBCRYPTO_SUPPORT
             libunwind: [ on  ]  # HAVE_LIBUNWIND_SUPPORT
    libdw-dwarf-unwind: [ on  ]  # HAVE_DWARF_SUPPORT
                  zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
                  lzma: [ on  ]  # HAVE_LZMA_SUPPORT
             get_cpuid: [ on  ]  # HAVE_AUXTRACE_SUPPORT
                   bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
                   aio: [ on  ]  # HAVE_AIO_SUPPORT
                  zstd: [ on  ]  # HAVE_ZSTD_SUPPORT
               libpfm4: [ OFF ]  # HAVE_LIBPFM
         libtraceevent: [ on  ]  # HAVE_LIBTRACEEVENT
root@roc-rk3399-pc:~#
root@roc-rk3399-pc:~# head -8 /proc/cpuinfo
processor	: 0
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4
root@roc-rk3399-pc:~# uname -a
Linux roc-rk3399-pc 6.1.0-rc5-00123-g4dd7ff4a0311 #2 SMP PREEMPT Wed Nov 16 19:55:11 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
root@roc-rk3399-pc:~#

So these events are supported there, lets see with something else..

Before:

root@roc-rk3399-pc:~# perf stat -a --bpf-counters -e LLC-loads,LLC-stores --for-each-cgroup system.slice,user.slice sleep 1
Failed to open first cgroup events
root@roc-rk3399-pc:~#

After:

root@roc-rk3399-pc:~# perf stat -a --bpf-counters -e LLC-loads,LLC-stores --for-each-cgroup system.slice,user.slice sleep 1

 Performance counter stats for 'system wide':

   <not supported>      LLC-loads                        system.slice                                          
   <not supported>      LLC-stores                       system.slice                                          
   <not supported>      LLC-loads                        user.slice                                            
   <not supported>      LLC-stores                       user.slice                                            

       1.016196455 seconds time elapsed

root@roc-rk3399-pc:~# 

And with mixed supported/unsupported:

root@roc-rk3399-pc:~# perf stat -a --bpf-counters -e LLC-loads,LLC-stores,L1-icache-loads,L1-dcache-loads --for-each-cgroup system.slice,user.slice sleep 1

 Performance counter stats for 'system wide':

   <not supported>      LLC-loads                        system.slice                                          
   <not supported>      LLC-stores                       system.slice                                          
            180903      L1-icache-loads                  system.slice                                          
             66861      L1-dcache-loads                  system.slice                                          
   <not supported>      LLC-loads                        user.slice                                            
   <not supported>      LLC-stores                       user.slice                                            
           2948290      L1-icache-loads                  user.slice                                            
           1962675      L1-dcache-loads                  user.slice                                            

       1.016265003 seconds time elapsed

root@roc-rk3399-pc:~#

Applied,

- Arnaldo
 
> After:
>   $ sudo ./perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads \
>   > --for-each-cgroup system.slice,user.slice  sleep 1
> 
>    Performance counter stats for 'system wide':
> 
>      <not supported>      L1-icache-loads                  system.slice
>           29,892,418      L1-dcache-loads                  system.slice
>      <not supported>      L1-icache-loads                  user.slice
>           52,497,220      L1-dcache-loads                  user.slice
> 
> Fixes: 944138f048f7d ("perf stat: Enable BPF counter with --for-each-cgroup")
> Signed-off-by: Namhyung Kim <namhyung@kernel.org>
> ---
>  tools/perf/util/bpf_counter_cgroup.c | 14 +++-----------
>  1 file changed, 3 insertions(+), 11 deletions(-)
> 
> diff --git a/tools/perf/util/bpf_counter_cgroup.c b/tools/perf/util/bpf_counter_cgroup.c
> index 3c2df7522f6f..1c82377ed78b 100644
> --- a/tools/perf/util/bpf_counter_cgroup.c
> +++ b/tools/perf/util/bpf_counter_cgroup.c
> @@ -116,27 +116,19 @@ static int bperf_load_program(struct evlist *evlist)
>  
>  			/* open single copy of the events w/o cgroup */
>  			err = evsel__open_per_cpu(evsel, evsel->core.cpus, -1);
> -			if (err) {
> -				pr_err("Failed to open first cgroup events\n");
> -				goto out;
> -			}
> +			if (err == 0)
> +				evsel->supported = true;
>  
>  			map_fd = bpf_map__fd(skel->maps.events);
>  			perf_cpu_map__for_each_cpu(cpu, j, evsel->core.cpus) {
>  				int fd = FD(evsel, j);
>  				__u32 idx = evsel->core.idx * total_cpus + cpu.cpu;
>  
> -				err = bpf_map_update_elem(map_fd, &idx, &fd,
> -							  BPF_ANY);
> -				if (err < 0) {
> -					pr_err("Failed to update perf_event fd\n");
> -					goto out;
> -				}
> +				bpf_map_update_elem(map_fd, &idx, &fd, BPF_ANY);
>  			}
>  
>  			evsel->cgrp = leader_cgrp;
>  		}
> -		evsel->supported = true;
>  
>  		if (evsel->cgrp == cgrp)
>  			continue;
> -- 
> 2.39.0.314.g84b9a713c41-goog

-- 

- Arnaldo

Re: [PATCH 4/4] perf stat: Do not use the same cgroup more than once

[Arnaldo Carvalho de Melo]

Em Tue, Jan 03, 2023 at 10:44:02PM -0800, Namhyung Kim escreveu:
> The --for-each-cgroup can have the same cgroup multiple times, but it makes
> bpf counters confused (since they have the same cgroup id), and the last
> cgroup events are counted only.  Let's check the cgroup name before adding
> a new entry.
> 
> Before:
>   $ sudo ./perf stat -a --bpf-counters --for-each-cgroup /,/ sleep 1
> 
>    Performance counter stats for 'system wide':
> 
>        <not counted> msec cpu-clock                        /
>        <not counted>      context-switches                 /
>        <not counted>      cpu-migrations                   /
>        <not counted>      page-faults                      /
>        <not counted>      cycles                           /
>        <not counted>      instructions                     /
>        <not counted>      branches                         /
>        <not counted>      branch-misses                    /
>             8,016.04 msec cpu-clock                        /                #    7.998 CPUs utilized
>                6,152      context-switches                 /                #  767.461 /sec
>                  250      cpu-migrations                   /                #   31.187 /sec
>                  442      page-faults                      /                #   55.139 /sec
>          613,111,487      cycles                           /                #    0.076 GHz
>          280,599,604      instructions                     /                #    0.46  insn per cycle
>           57,692,724      branches                         /                #    7.197 M/sec
>            3,385,168      branch-misses                    /                #    5.87% of all branches
> 
>          1.002220125 seconds time elapsed
> 
> After:
>   $ sudo ./perf stat -a --bpf-counters --for-each-cgroup /,/  sleep 1
> 
>    Performance counter stats for 'system wide':
> 
>             8,013.38 msec cpu-clock                        /                #    7.998 CPUs utilized
>                6,859      context-switches                 /                #  855.944 /sec
>                  334      cpu-migrations                   /                #   41.680 /sec
>                  345      page-faults                      /                #   43.053 /sec
>          782,326,119      cycles                           /                #    0.098 GHz
>          471,645,724      instructions                     /                #    0.60  insn per cycle
>           94,963,430      branches                         /                #   11.851 M/sec
>            3,685,511      branch-misses                    /                #    3.88% of all branches
> 
>          1.001864539 seconds time elapsed
> 
> Fixes: bb1c15b60b981 ("perf stat: Support regex pattern in --for-each-cgroup")
> Signed-off-by: Namhyung Kim <namhyung@kernel.org>

Tested and appied.

- Arnaldo

> ---
>  tools/perf/util/cgroup.c | 23 ++++++++++++++++++-----
>  1 file changed, 18 insertions(+), 5 deletions(-)
> 
> diff --git a/tools/perf/util/cgroup.c b/tools/perf/util/cgroup.c
> index e99b41f9be45..cd978c240e0d 100644
> --- a/tools/perf/util/cgroup.c
> +++ b/tools/perf/util/cgroup.c
> @@ -224,6 +224,19 @@ static int add_cgroup_name(const char *fpath, const struct stat *sb __maybe_unus
>  	return 0;
>  }
>  
> +static int check_and_add_cgroup_name(const char *fpath)
> +{
> +	struct cgroup_name *cn;
> +
> +	list_for_each_entry(cn, &cgroup_list, list) {
> +		if (!strcmp(cn->name, fpath))
> +			return 0;
> +	}
> +
> +	/* pretend if it's added by ftw() */
> +	return add_cgroup_name(fpath, NULL, FTW_D, NULL);
> +}
> +
>  static void release_cgroup_list(void)
>  {
>  	struct cgroup_name *cn;
> @@ -242,7 +255,7 @@ static int list_cgroups(const char *str)
>  	struct cgroup_name *cn;
>  	char *s;
>  
> -	/* use given name as is - for testing purpose */
> +	/* use given name as is when no regex is given */
>  	for (;;) {
>  		p = strchr(str, ',');
>  		e = p ? p : eos;
> @@ -253,13 +266,13 @@ static int list_cgroups(const char *str)
>  			s = strndup(str, e - str);
>  			if (!s)
>  				return -1;
> -			/* pretend if it's added by ftw() */
> -			ret = add_cgroup_name(s, NULL, FTW_D, NULL);
> +
> +			ret = check_and_add_cgroup_name(s);
>  			free(s);
> -			if (ret)
> +			if (ret < 0)
>  				return -1;
>  		} else {
> -			if (add_cgroup_name("", NULL, FTW_D, NULL) < 0)
> +			if (check_and_add_cgroup_name("/") < 0)
>  				return -1;
>  		}
>  
> -- 
> 2.39.0.314.g84b9a713c41-goog

-- 

- Arnaldo

Re: [PATCH 0/4] perf bpf_counter: A set of random fixes (v1)

[Arnaldo Carvalho de Melo]

Em Tue, Jan 03, 2023 at 10:43:58PM -0800, Namhyung Kim escreveu:
> Hello,
> 
> This is a collection of small fixes for perf stat bpf counters (bperf).
> 
> The bperf framework maintains perf_attr_map in the BPF fs to share the
> same event as much as possible.  But the size was limited to 16 and
> perf stat with -ddd option would create more than 16 events and fails.
> 
> Also cgroup events with --for-each-cgroup had some other problems when
> dealing with unsupported events and duplicate cgroups.
> 
> The code is available at 'perf/stat-bpf-fix-v1' branch in
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

The first two are improvements, not strict fixes, so I'm leaving them
for the next merge window.

- Arnaldo
 
> Thanks,
> Namhyung
> 
> Namhyung Kim (4):
>   perf bpf_counter: Add more error messages for bperf
>   perf bpf_counter: Increase perf_attr_map entries to 32
>   perf bpf_counter: Handle unsupported cgroup events
>   perf stat: Do not use the same cgroup more than once
> 
>  tools/perf/util/bpf_counter.c        | 11 ++++++++---
>  tools/perf/util/bpf_counter_cgroup.c | 14 +++-----------
>  tools/perf/util/cgroup.c             | 23 ++++++++++++++++++-----
>  3 files changed, 29 insertions(+), 19 deletions(-)
> 
> 
> base-commit: d8d85ce86dc82de4f88b821a78f533b9d5b22a45
> -- 
> 2.39.0.314.g84b9a713c41-goog

-- 

- Arnaldo