* Re: [PATCH 00/34] perf clang: Builtin clang and perfhook support
@ 2016-11-15 4:57 Alexei Starovoitov
2016-11-15 5:03 ` Wangnan (F)
0 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2016-11-15 4:57 UTC (permalink / raw)
To: Wang Nan
Cc: Arnaldo Carvalho de Melo, Alexei Starovoitov, Zefan Li, He Kuang,
linux-kernel, pi3orama, Jiri Olsa
On Mon, Nov 14, 2016 at 8:05 PM, Wang Nan <wangnan0@huawei.com> wrote:
> This is version 2 of perf builtin clang patch series. Compare to v1,
> add an exciting feature: jit compiling perf hook functions. This
> features allows script writer report result through BPF map in a
> customized way.
looks great.
> SEC("perfhook:record_start")
> void record_start(void *ctx)
> {
> int perf_pid = getpid(), key = G_perf_pid;
> printf("Start count, perfpid=%d\n", perf_pid);
> jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
the name, I think, is too verbose.
Why not to keep them as bpf_map_update_elem
even for user space programs?
> SEC("perfhook:record_end")
> void record_end(void *ctx)
> {
> u64 key = -1, value;
> while (!jit_helper__map_get_next_key(ctx, &syscall_counter, &key, &key)) {
> jit_helper__map_lookup_elem(ctx, &syscall_counter, &key, &value);
> printf("syscall %ld\tcount: %ld\n", (long)key, (long)value);
this loop will be less verbose as well.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/34] perf clang: Builtin clang and perfhook support
2016-11-15 4:57 [PATCH 00/34] perf clang: Builtin clang and perfhook support Alexei Starovoitov
@ 2016-11-15 5:03 ` Wangnan (F)
0 siblings, 0 replies; 6+ messages in thread
From: Wangnan (F) @ 2016-11-15 5:03 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Arnaldo Carvalho de Melo, Alexei Starovoitov, Zefan Li, He Kuang,
linux-kernel, pi3orama, Jiri Olsa
On 2016/11/15 12:57, Alexei Starovoitov wrote:
> On Mon, Nov 14, 2016 at 8:05 PM, Wang Nan <wangnan0@huawei.com> wrote:
>> This is version 2 of perf builtin clang patch series. Compare to v1,
>> add an exciting feature: jit compiling perf hook functions. This
>> features allows script writer report result through BPF map in a
>> customized way.
> looks great.
>
>> SEC("perfhook:record_start")
>> void record_start(void *ctx)
>> {
>> int perf_pid = getpid(), key = G_perf_pid;
>> printf("Start count, perfpid=%d\n", perf_pid);
>> jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
> the name, I think, is too verbose.
> Why not to keep them as bpf_map_update_elem
> even for user space programs?
I can make it shorter by give it a better name or use a wrapper like
BPF_MAP(update_elem)
but the only thing I can't do is to make perfhook and in-kernel script
use a uniform name for these bpf_map functions, because
bpf_map_update_elem is already defined:
"static long (*bpf_map_update_elem)(void *, void *, void *, unsigned
long) = (void *)2;\n"
>> SEC("perfhook:record_end")
>> void record_end(void *ctx)
>> {
>> u64 key = -1, value;
>> while (!jit_helper__map_get_next_key(ctx, &syscall_counter, &key, &key)) {
>> jit_helper__map_lookup_elem(ctx, &syscall_counter, &key, &value);
>> printf("syscall %ld\tcount: %ld\n", (long)key, (long)value);
> this loop will be less verbose as well.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/34] perf clang: Builtin clang and perfhook support
2016-11-15 5:21 Alexei Starovoitov
@ 2016-11-15 5:37 ` Wangnan (F)
0 siblings, 0 replies; 6+ messages in thread
From: Wangnan (F) @ 2016-11-15 5:37 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Arnaldo Carvalho de Melo, Alexei Starovoitov, Zefan Li, He Kuang,
linux-kernel, pi3orama, Jiri Olsa
On 2016/11/15 13:21, Alexei Starovoitov wrote:
> On Mon, Nov 14, 2016 at 9:03 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>>
>> On 2016/11/15 12:57, Alexei Starovoitov wrote:
>>> On Mon, Nov 14, 2016 at 8:05 PM, Wang Nan <wangnan0@huawei.com> wrote:
>>>> This is version 2 of perf builtin clang patch series. Compare to v1,
>>>> add an exciting feature: jit compiling perf hook functions. This
>>>> features allows script writer report result through BPF map in a
>>>> customized way.
>>> looks great.
>>>
>>>> SEC("perfhook:record_start")
>>>> void record_start(void *ctx)
>>>> {
>>>> int perf_pid = getpid(), key = G_perf_pid;
>>>> printf("Start count, perfpid=%d\n", perf_pid);
>>>> jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
>>> the name, I think, is too verbose.
>>> Why not to keep them as bpf_map_update_elem
>>> even for user space programs?
>>
>> I can make it shorter by give it a better name or use a wrapper like
>>
>> BPF_MAP(update_elem)
> the macro isn't pretty, since function calls won't look like calls.
>
>> but the only thing I can't do is to make perfhook and in-kernel script
>> use a uniform name for these bpf_map functions, because
>> bpf_map_update_elem is already defined:
>>
>> "static long (*bpf_map_update_elem)(void *, void *, void *, unsigned long) =
>> (void *)2;\n"
> right. i guess you could have #ifdef it, so it's different for bpf backend
> and for native.
Then the '.c' -> LLVM IR compiling should be done twice for BPF
and for JIT to make the macro work. In current implementation
we have only one LLVM IR. It is faster and can make sure the data
layout ("maps" section) is identical.
> Another alternative is to call it map_update_elem or map_update
> or bpf_map_update. Something shorter is already a win.
> 'jit_helper__' prefix is an implementation detail. The users don't
> need to know and don't need to spell it out everywhere.
Good. Let choose a better name for them.
Thank you.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/34] perf clang: Builtin clang and perfhook support
@ 2016-11-15 5:21 Alexei Starovoitov
2016-11-15 5:37 ` Wangnan (F)
0 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2016-11-15 5:21 UTC (permalink / raw)
To: Wangnan (F)
Cc: Arnaldo Carvalho de Melo, Alexei Starovoitov, Zefan Li, He Kuang,
linux-kernel, pi3orama, Jiri Olsa
On Mon, Nov 14, 2016 at 9:03 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>
>
> On 2016/11/15 12:57, Alexei Starovoitov wrote:
>>
>> On Mon, Nov 14, 2016 at 8:05 PM, Wang Nan <wangnan0@huawei.com> wrote:
>>>
>>> This is version 2 of perf builtin clang patch series. Compare to v1,
>>> add an exciting feature: jit compiling perf hook functions. This
>>> features allows script writer report result through BPF map in a
>>> customized way.
>>
>> looks great.
>>
>>> SEC("perfhook:record_start")
>>> void record_start(void *ctx)
>>> {
>>> int perf_pid = getpid(), key = G_perf_pid;
>>> printf("Start count, perfpid=%d\n", perf_pid);
>>> jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
>>
>> the name, I think, is too verbose.
>> Why not to keep them as bpf_map_update_elem
>> even for user space programs?
>
>
> I can make it shorter by give it a better name or use a wrapper like
>
> BPF_MAP(update_elem)
the macro isn't pretty, since function calls won't look like calls.
> but the only thing I can't do is to make perfhook and in-kernel script
> use a uniform name for these bpf_map functions, because
> bpf_map_update_elem is already defined:
>
> "static long (*bpf_map_update_elem)(void *, void *, void *, unsigned long) =
> (void *)2;\n"
right. i guess you could have #ifdef it, so it's different for bpf backend
and for native.
Another alternative is to call it map_update_elem or map_update
or bpf_map_update. Something shorter is already a win.
'jit_helper__' prefix is an implementation detail. The users don't
need to know and don't need to spell it out everywhere.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/34] perf clang: Builtin clang and perfhook support
2016-11-15 4:05 Wang Nan
@ 2016-11-15 4:32 ` Wangnan (F)
0 siblings, 0 replies; 6+ messages in thread
From: Wangnan (F) @ 2016-11-15 4:32 UTC (permalink / raw)
To: acme, ast; +Cc: lizefan, hekuang, linux-kernel, pi3orama, Jiri Olsa
On 2016/11/15 12:05, Wang Nan wrote:
> $ sudo -s
> # ulimit -l unlimited
> # perf record -e ./count_syscalls.c echo "Haha"
> Start count, perfpid=25209
> Haha
> [ perf record: Woken up 1 times to write data ]
> syscall 8 count: 6
> syscall 11 count: 1
> syscall 4 count: 6
> syscall 21 count: 1
> syscall 5 count: 3
> syscall 231 count: 1
> syscall 45 count: 3
> syscall 0 count: 24
> syscall 257 count: 1
> syscall 59 count: 4
> syscall 23 count: 9
> syscall 78 count: 2
> syscall 41 count: 4
> syscall 72 count: 8
> syscall 10 count: 3
> syscall 321 count: 1
> syscall 298 count: 7
> syscall 16 count: 21
> syscall 9 count: 16
> syscall 1 count: 114
> syscall 12 count: 3
> syscall 14 count: 35
> syscall 158 count: 1
> syscall 2 count: 15
> syscall 7 count: 18
> syscall 3 count: 11
> [ perf record: Captured and wrote 0.011 MB perf.data ]
Note that this example counts system wide syscall histogram, not
only 'echo' proc. The in-kernel BPF script doesn't know pid of 'echo'
so can't filter base on it. I'm planning adding more perf hook points
to pass information like this.
Thank you.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 00/34] perf clang: Builtin clang and perfhook support
@ 2016-11-15 4:05 Wang Nan
2016-11-15 4:32 ` Wangnan (F)
0 siblings, 1 reply; 6+ messages in thread
From: Wang Nan @ 2016-11-15 4:05 UTC (permalink / raw)
To: acme, ast; +Cc: lizefan, hekuang, linux-kernel, pi3orama, Wang Nan, Jiri Olsa
This is version 2 of perf builtin clang patch series. Compare to v1,
add an exciting feature: jit compiling perf hook functions. This
features allows script writer report result through BPF map in a
customized way.
At the end of this cover letter lists an example shows how to capture
and report a syscall histogram.
This patchset is based on current perf/core.
In this patchset:
Patch 1 - 4 are bugfixes left in my local tree.
Patch 5 - 7 are preparation in libbpf.
Patch 8 - 9 introduce perf hook to perf.
Patch 10 - 21 are patches support builtin clang. Some of then are already
collected by Arnaldo in tmp.perf/builtin-clang. They are slightly
adjusted in this v2 series.
Patch 22 - 29 add JIT compiling to builting clang and add them to perf hook.
Patch 30 - 34 are easy of use improvements. 1) builtin clang defines macros
and helpers by default using builtin include headers. Default
headers can be turned off by -UBUILTIN_CLANG_DEFAULT_INCLUDE.
2) allow JITted perf hooks access more functions in libc.
Example: build syscall histogram, exclude syscalls issued by perf itself.
Please note following improvements:
1. Don't need define many bpf helpers by hand.
See bpf_map_lookup_elem and bpf_get_current_pid_tgid.
2. See how pid of git is passed to BPF script attached to
raw_syscalls:sys_enter.
$ cat ./count_syscalls.c
typedef unsigned long u64;
#define BPF_MAP_TYPE_HASH 1
#define BPF_MAP_TYPE_ARRAY 2
enum GVAL {
G_perf_pid,
NR_GVALS
};
struct bpf_map_def SEC("maps") GVALS = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(u64),
.max_entries = NR_GVALS,
};
struct bpf_map_def SEC("maps") syscall_counter = {
.type = BPF_MAP_TYPE_HASH,
.key_size = sizeof(u64),
.value_size = sizeof(u64),
.max_entries = 512,
};
SEC("raw_syscalls:sys_enter")
int func(void *ctx)
{
int key = G_perf_pid;
u64 id = *((u64 *)(ctx + 8));
int self_pid = bpf_get_current_pid_tgid() & 0xffffffff;
int *perf_pid = bpf_map_lookup_elem(&GVALS, &key);
u64 *counter;
if (!perf_pid)
return 0;
if (*perf_pid == self_pid)
return 0;
counter = bpf_map_lookup_elem(&syscall_counter, &id);
if (!counter) {
u64 value = 1;
bpf_map_update_elem(&syscall_counter, &id, &value, 0);
return 0;
}
__sync_fetch_and_add(counter, 1);
return 0;
}
SEC("perfhook:record_start")
void record_start(void *ctx)
{
int perf_pid = getpid(), key = G_perf_pid;
printf("Start count, perfpid=%d\n", perf_pid);
jit_helper__map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
}
SEC("perfhook:record_end")
void record_end(void *ctx)
{
u64 key = -1, value;
while (!jit_helper__map_get_next_key(ctx, &syscall_counter, &key, &key)) {
jit_helper__map_lookup_elem(ctx, &syscall_counter, &key, &value);
printf("syscall %ld\tcount: %ld\n", (long)key, (long)value);
}
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
$ sudo -s
# ulimit -l unlimited
# perf record -e ./count_syscalls.c echo "Haha"
Start count, perfpid=25209
Haha
[ perf record: Woken up 1 times to write data ]
syscall 8 count: 6
syscall 11 count: 1
syscall 4 count: 6
syscall 21 count: 1
syscall 5 count: 3
syscall 231 count: 1
syscall 45 count: 3
syscall 0 count: 24
syscall 257 count: 1
syscall 59 count: 4
syscall 23 count: 9
syscall 78 count: 2
syscall 41 count: 4
syscall 72 count: 8
syscall 10 count: 3
syscall 321 count: 1
syscall 298 count: 7
syscall 16 count: 21
syscall 9 count: 16
syscall 1 count: 114
syscall 12 count: 3
syscall 14 count: 35
syscall 158 count: 1
syscall 2 count: 15
syscall 7 count: 18
syscall 3 count: 11
[ perf record: Captured and wrote 0.011 MB perf.data ]
Eric Leblond (1):
tools lib bpf: fix maps resolution
Wang Nan (33):
perf tools: Fix kernel version error in ubuntu
perf record: Fix segfault when running with suid and kptr_restrict is
1
tools perf: Add missing struct defeinition in probe_event.h
tools lib bpf: Add missing bpf map functions
tools lib bpf: Add private field for bpf_object
tools lib bpf: Retrive bpf_map through offset of bpf_map_def
perf tools: Introduce perf hooks
perf tools: Pass context to perf hook functions
perf llvm: Extract helpers in llvm-utils.c
tools build: Add feature detection for LLVM
tools build: Add feature detection for clang
perf build: Add clang and llvm compile and linking support
perf clang: Add builtin clang support ant test case
perf clang: Use real file system for #include
perf clang: Allow passing CFLAGS to builtin clang
perf clang: Update test case to use real BPF script
perf clang: Support compile IR to BPF object and add testcase
perf clang: Compile BPF script use builtin clang support
perf clang: Pass full path to builtin clang
perf clang: Pass CFLAGS to builtin clang
perf clang jit: Wrap llvm::Module using PerfModule
perf clang jit: Insignt BPF and JIT functions in a Module
perf clang jit: add PerfModule::doJIT to JIT perfhook functions.
perf clang jit: Export functions for jitted code
perf clang jit: Actually JIT and hook in bpf loader
perf clang jit: Collect the lowest address in maps section as map_base
perf clang jit: Access BPF map
perf clang jit: Allow jitted perf hook access BPF maps
perf clang: Link BPF functions declaration into perf
perf clang: Declare BPF functions for BPF scripts automatically
perf clang: Include helpers to BPF scripts
perf clang builtin: Define hook helpers by default
perf clang git: Export getpid() to perf hook
tools/build/feature/Makefile | 18 +
tools/build/feature/test-clang.cpp | 21 ++
tools/build/feature/test-llvm.cpp | 8 +
tools/lib/bpf/bpf.c | 56 +++
tools/lib/bpf/bpf.h | 7 +
tools/lib/bpf/libbpf.c | 177 ++++++---
tools/lib/bpf/libbpf.h | 13 +
tools/perf/Makefile.config | 62 +++-
tools/perf/Makefile.perf | 23 +-
tools/perf/builtin-record.c | 11 +
tools/perf/tests/Build | 4 +-
tools/perf/tests/bpf-script-example.c | 30 +-
tools/perf/tests/bpf-script-test-kbuild.c | 2 +
tools/perf/tests/bpf-script-test-prologue.c | 6 +-
tools/perf/tests/bpf-script-test-relocation.c | 17 +-
tools/perf/tests/builtin-test.c | 13 +
tools/perf/tests/clang.c | 50 +++
tools/perf/tests/llvm.h | 7 +
tools/perf/tests/make | 2 +
tools/perf/tests/perf-hooks.c | 48 +++
tools/perf/tests/tests.h | 4 +
tools/perf/util/Build | 5 +
tools/perf/util/bpf-loader.c | 102 +++++-
tools/perf/util/bpf-loader.h | 19 +
tools/perf/util/c++/Build | 4 +
tools/perf/util/c++/bpf-funcs-str.c | 228 ++++++++++++
tools/perf/util/c++/bpf-helper-str.c | 24 ++
tools/perf/util/c++/clang-bpf-includes.h | 13 +
tools/perf/util/c++/clang-c.h | 63 ++++
tools/perf/util/c++/clang-test.cpp | 99 +++++
tools/perf/util/c++/clang.cpp | 498 ++++++++++++++++++++++++++
tools/perf/util/c++/clang.h | 61 ++++
tools/perf/util/jit-helpers.c | 60 ++++
tools/perf/util/jit-helpers.h | 30 ++
tools/perf/util/llvm-utils.c | 76 +++-
tools/perf/util/llvm-utils.h | 15 +-
tools/perf/util/perf-hooks-list.h | 3 +
tools/perf/util/perf-hooks.c | 88 +++++
tools/perf/util/perf-hooks.h | 39 ++
tools/perf/util/probe-event.h | 2 +
tools/perf/util/symbol.c | 2 +-
tools/perf/util/util-cxx.h | 26 ++
tools/perf/util/util.c | 55 ++-
43 files changed, 1991 insertions(+), 100 deletions(-)
create mode 100644 tools/build/feature/test-clang.cpp
create mode 100644 tools/build/feature/test-llvm.cpp
create mode 100644 tools/perf/tests/clang.c
create mode 100644 tools/perf/tests/perf-hooks.c
create mode 100644 tools/perf/util/c++/Build
create mode 100644 tools/perf/util/c++/bpf-funcs-str.c
create mode 100644 tools/perf/util/c++/bpf-helper-str.c
create mode 100644 tools/perf/util/c++/clang-bpf-includes.h
create mode 100644 tools/perf/util/c++/clang-c.h
create mode 100644 tools/perf/util/c++/clang-test.cpp
create mode 100644 tools/perf/util/c++/clang.cpp
create mode 100644 tools/perf/util/c++/clang.h
create mode 100644 tools/perf/util/jit-helpers.c
create mode 100644 tools/perf/util/jit-helpers.h
create mode 100644 tools/perf/util/perf-hooks-list.h
create mode 100644 tools/perf/util/perf-hooks.c
create mode 100644 tools/perf/util/perf-hooks.h
create mode 100644 tools/perf/util/util-cxx.h
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
--
2.10.1
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-11-15 5:38 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-15 4:57 [PATCH 00/34] perf clang: Builtin clang and perfhook support Alexei Starovoitov
2016-11-15 5:03 ` Wangnan (F)
-- strict thread matches above, loose matches on Subject: below --
2016-11-15 5:21 Alexei Starovoitov
2016-11-15 5:37 ` Wangnan (F)
2016-11-15 4:05 Wang Nan
2016-11-15 4:32 ` Wangnan (F)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).