Re: [PATCH 3/3] perf stat: Enable BPF counter with --for-each-cgroup

[Namhyung Kim <namhyung@kernel.org>]

On Thu, Jun 24, 2021 at 2:41 PM Song Liu <songliubraving@fb.com> wrote:
>
>
>
> > On Jun 24, 2021, at 2:01 PM, Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > On Thu, Jun 24, 2021 at 9:20 AM Song Liu <songliubraving@fb.com> wrote:
> >>>>> +
> >>>>> +// single set of global perf events to measure
> >>>>> +struct {
> >>>>> +     __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
> >>>>> +     __uint(key_size, sizeof(__u32));
> >>>>> +     __uint(value_size, sizeof(int));
> >>>>> +     __uint(max_entries, 1);
> >>>>> +} events SEC(".maps");
> >>>>> +
> >>>>> +// from logical cpu number to event index
> >>>>> +// useful when user wants to count subset of cpus
> >>>>> +struct {
> >>>>> +     __uint(type, BPF_MAP_TYPE_HASH);
> >>>>> +     __uint(key_size, sizeof(__u32));
> >>>>> +     __uint(value_size, sizeof(__u32));
> >>>>> +     __uint(max_entries, 1);
> >>>>> +} cpu_idx SEC(".maps");
> >>>>
> >>>> How about we make cpu_idx a percpu array and use 0,1 for
> >>>> disable/enable profiling on this cpu?
> >>>
> >>> No, it's to calculate an index to the cgrp_readings map which
> >>> has the event x cpu x cgroup number of elements.
> >>>
> >>> It controls enabling events with a global (bss) variable.
> >>
> >> If we make cgrp_idx a per cpu array, we probably don't need the
> >> cpu_idx map?
> >
> > Right.

Maybe not.  Sometimes we want to profile a subset of cpus only.
In that case, cpu != idx then I think we still need this.


> >
> >>
> >>>
> >>>>
> >>>>> +
> >>>>> +// from cgroup id to event index
> >>>>> +struct {
> >>>>> +     __uint(type, BPF_MAP_TYPE_HASH);
> >>>>> +     __uint(key_size, sizeof(__u64));
> >>>>> +     __uint(value_size, sizeof(__u32));
> >>>>> +     __uint(max_entries, 1);
> >>>>> +} cgrp_idx SEC(".maps");
> >>>>> +
> >>>>> +// per-cpu event snapshots to calculate delta
> >>>>> +struct {
> >>>>> +     __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
> >>>>> +     __uint(key_size, sizeof(__u32));
> >>>>> +     __uint(value_size, sizeof(struct bpf_perf_event_value));
> >>>>> +} prev_readings SEC(".maps");
> >>>>> +
> >>>>> +// aggregated event values for each cgroup
> >>>>> +// will be read from the user-space
> >>>>> +struct {
> >>>>> +     __uint(type, BPF_MAP_TYPE_ARRAY);
> >>>>> +     __uint(key_size, sizeof(__u32));
> >>>>> +     __uint(value_size, sizeof(struct bpf_perf_event_value));
> >>>>> +} cgrp_readings SEC(".maps");
> >>>>
> >>>> Maybe also make this a percpu array? This should make the BPF program
> >>>> faster.
> >>>
> >>> Maybe.  But I don't know how to access the elements
> >>> in a per-cpu map from userspace.
> >>
> >> Please refer to bperf__read() reading accum_readings. Basically, we read
> >> one index of all CPUs with one bpf_map_lookup_elem().
> >
> > Thanks!  So when I use a per-cpu array with 3 elements, I can access
> > to cpu/elem entries in a row like below, right?
> >
> >  0/0, 0/1, 0/2, 1/0, 1/1, 1/2, 2/0, 2/1, 2/2, 3/0, ...
>
> I am not sure I am following here.
>
> Say the system have 10 cpus, and the array has 3 elements. We can do:
>
>         __u32 values[10];  /* assuming both key and value are __u32 */
>         __u32 elem;
>         int cpu;
>
>         for (elem = 0; elem < 3; elem++) {
>                 bpf_map_lookup_elem(map_fd, &elem, values);
>                 for (cpu = 0; cpu < 10; cpu++)
>                         values[cpu] /* this is the value for cpu/elem */
>         }

Thanks for the explanation, I didn't think that way.
I thought it like below:

    __u32 elem, value;

    for (elem = 0; elem < 3 * 10; elem++) {
        bpf_map_lookup_elem(map_fd, &elem, &value);
    }

So in this case, the actual value size is like below, right?

  value-size = map-value-size * number-of-cpu

Thanks,
Namhyung