Re: [RFC PATCH v6 00/32] perf tools: filtering events using eBPF programs

["Wangnan (F)" <wangnan0@huawei.com>]

On 2015/6/9 21:59, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jun 09, 2015 at 05:50:04AM +0000, Wang Nan escreveu:
>> This is the 6th version which tries to introduce eBPF programs to perf.
>> It enables 'perf record' to filter events using eBPF programs like:
>>
>>   # perf record --event bpf-file.c sleep 1
> Thanks for doing this, comments below for this 00/32 file:
>   
>> and
>>
>>   # perf record --event bpf-file.o sleep 1
>>
>> This patch series is based on tip/perf/core (a3d8654).
>>
>> Compared with v5 patch, this series allows user to pass BPF
>> scriptlet source code file directly, instead of requiring user to
>> pre-compile it into an ELF object. Which is done mainly by patch 24/32.
>>
>> In addition, to help users setup LLVM, [bpf] perf config section is
>> appended. 4 command line options are also appended to 'perf record'
>> for setting pathes and options of clang and llc. Which is done by patch
>> 25/32 and 32/32.
>>
>> Other improvements:
>>
>>   1. Adjust context.
>>
>>   2. Get rid of 'trailing whitespace' and 'space before tab in indent'
>>      problems.
>>
>>   3. Error messages are improved when BPF API checking fail.
>>
>> In this series:
>>
>>   Patch 1/32 adds a feature check to check version of eBPF API.
>>
>>   Patch 2/32 - 21/32 introduce libbpf, which first parse eBPF object
>>     files then load maps and programs into kernel. These patches are
>>     already received 'Acked-by' from Alexei Starovoitov except patch
>>     5/32, which enables opening a memory-based object file image using
>>     bpf_object__open_buffer().
>>
>>   Patch 22/32 - 32/32 are perf related. Patch 24, 25 and 32 are new to
>>     support dynamical compiling.
>>
>> Here is an example shows the using of dynamic compiling:
>>
>>     # perf record --event lock_page.c ls /
>>     Added new event:
>>       perf_bpf_probe:lock_page (on __lock_page)
>>     
>>     You can now use it in all perf tools, such as:
> Can we suppress these messages? They are misleading, because we will not
> be able to " use it in all perf tools", since they will be removed right
> after this ' perf record'  session ends.
>
> They may have value as debugging aid, so that we see what probes are
> being put in place, etc, but for the default case they are just annoying
> scrolling.

Sure. I don't want to alert API of add_perf_probe_events() and 
__add_probe_trace_events(),
so in next version I'd like to introduce a new field in probe_conf to 
control these output.

>>     	perf record -e perf_bpf_probe:lock_page -aR sleep 1
>>     
>>     Added new event:
>>       perf_bpf_probe:lock_page_ret (on __lock_page%return)
>>     
>>     You can now use it in all perf tools, such as:
>>     
>>     	perf record -e perf_bpf_probe:lock_page_ret -aR sleep 1
>>     
>>     Added new event:
>>       perf_bpf_probe:unlock_page (on unlock_page)
>>     
>>     You can now use it in all perf tools, such as:
>>     
>>     	perf record -e perf_bpf_probe:unlock_page -aR sleep 1
>>     
>>     bin  boot  dev	etc  home  init  lib  lib64  linuxrc  media  mnt  proc	root  run  sbin  sys  tmp  usr	var
>>     [ perf record: Woken up 1 times to write data ]
>>     [ perf record: Captured and wrote 0.002 MB perf.data ]
>>     Removed event: perf_bpf_probe:lock_page
>>     Removed event: perf_bpf_probe:lock_page_ret
>>     Removed event: perf_bpf_probe:unlock_page
> Ditto for this.
>   
>> Where lock_page.c can be found from:
>>
>>   http://lkml.kernel.org/r/557025A2.6090903@huawei.com
> I'll try applying the patches and running this script, perhaps it would
> be a good idea to have some /examples/  directory under som of the
> tools/ directory, so that everything that is needed to test this is in
> the source tree.
>
> We need to at some point, best if right now, to make 'perf test' have an
> entry for this, is this in this patchkit?

Will append. Tow parts: 1. testing llvm compiling BPF source code; 2. 
loading it into kernel.

> I.e. plain running 'perf test'  will tell us if all this works fine, now
> and everytime somebody with all the needed bits runs 'perf test'.
>
> Thanks for the hard work!
>
> - Arnaldo
>   
>> Wang Nan (32):
>>    tools build: Add feature check for eBPF API
>>    bpf tools: Introduce 'bpf' library to tools
>>    bpf tools: Allow caller to set printing function
>>    bpf tools: Open eBPF object file and do basic validation
>>    bpf tools: Read eBPF object from buffer
>>    bpf tools: Check endianess and make libbpf fail early
>>    bpf tools: Iterate over ELF sections to collect information
>>    bpf tools: Collect version and license from ELF sections
>>    bpf tools: Collect map definitions from 'maps' section
>>    bpf tools: Collect symbol table from SHT_SYMTAB section
>>    bpf tools: Collect eBPF programs from their own sections
>>    bpf tools: Collect relocation sections from SHT_REL sections
>>    bpf tools: Record map accessing instructions for each program
>>    bpf tools: Add bpf.c/h for common bpf operations
>>    bpf tools: Create eBPF maps defined in an object file
>>    bpf tools: Relocate eBPF programs
>>    bpf tools: Introduce bpf_load_program() to bpf.c
>>    bpf tools: Load eBPF programs in object files into kernel
>>    bpf tools: Introduce accessors for struct bpf_program
>>    bpf tools: Introduce accessors for struct bpf_object
>>    bpf tools: Link all bpf objects onto a list
>>    perf tools: Make perf depend on libbpf
>>    perf record: Enable passing bpf object file to --event
>>    perf record: Compile scriptlets if pass '.c' to --event
>>    perf tools: Add 'bpf.' config section to perf default config
>>    perf tools: Parse probe points of eBPF programs during preparation
>>    perf probe: Attach trace_probe_event with perf_probe_event
>>    perf record: Probe at kprobe points
>>    perf record: Load all eBPF object into kernel
>>    perf tools: Add bpf_fd field to evsel and config it
>>    perf tools: Attach eBPF program to perf event
>>    perf record: Add LLVM options for compiling BPF scripts
>>
>>   tools/build/Makefile.feature   |    6 +-
>>   tools/build/feature/Makefile   |    6 +-
>>   tools/build/feature/test-bpf.c |   18 +
>>   tools/lib/bpf/.gitignore       |    2 +
>>   tools/lib/bpf/Build            |    1 +
>>   tools/lib/bpf/Makefile         |  195 ++++++++
>>   tools/lib/bpf/bpf.c            |   84 ++++
>>   tools/lib/bpf/bpf.h            |   23 +
>>   tools/lib/bpf/libbpf.c         | 1037 ++++++++++++++++++++++++++++++++++++++++
>>   tools/lib/bpf/libbpf.h         |   85 ++++
>>   tools/perf/MANIFEST            |    3 +
>>   tools/perf/Makefile.perf       |   17 +-
>>   tools/perf/builtin-probe.c     |    2 +-
>>   tools/perf/builtin-record.c    |   40 +-
>>   tools/perf/config/Makefile     |   19 +-
>>   tools/perf/tests/make          |    4 +-
>>   tools/perf/util/Build          |    1 +
>>   tools/perf/util/bpf-loader.c   |  477 ++++++++++++++++++
>>   tools/perf/util/bpf-loader.h   |   54 +++
>>   tools/perf/util/config.c       |    3 +
>>   tools/perf/util/debug.c        |    5 +
>>   tools/perf/util/debug.h        |    1 +
>>   tools/perf/util/evlist.c       |   51 ++
>>   tools/perf/util/evlist.h       |    1 +
>>   tools/perf/util/evsel.c        |   17 +
>>   tools/perf/util/evsel.h        |    1 +
>>   tools/perf/util/parse-events.c |   16 +
>>   tools/perf/util/parse-events.h |    2 +
>>   tools/perf/util/parse-events.l |    6 +
>>   tools/perf/util/parse-events.y |   29 +-
>>   tools/perf/util/probe-event.c  |   57 ++-
>>   tools/perf/util/probe-event.h  |    5 +-
>>   32 files changed, 2229 insertions(+), 39 deletions(-)
>>   create mode 100644 tools/build/feature/test-bpf.c
>>   create mode 100644 tools/lib/bpf/.gitignore
>>   create mode 100644 tools/lib/bpf/Build
>>   create mode 100644 tools/lib/bpf/Makefile
>>   create mode 100644 tools/lib/bpf/bpf.c
>>   create mode 100644 tools/lib/bpf/bpf.h
>>   create mode 100644 tools/lib/bpf/libbpf.c
>>   create mode 100644 tools/lib/bpf/libbpf.h
>>   create mode 100644 tools/perf/util/bpf-loader.c
>>   create mode 100644 tools/perf/util/bpf-loader.h
>>
>> -- 
>> 1.8.3.4