From: Andrii Nakryiko <andrii@kernel.org>
To: <bpf@vger.kernel.org>, <ast@kernel.org>, <daniel@iogearbox.net>
Cc: <andrii@kernel.org>, <kernel-team@fb.com>,
Peter Zijlstra <peterz@infradead.org>
Subject: [PATCH v3 bpf-next 00/14] BPF perf link and user-provided bpf_cookie
Date: Thu, 29 Jul 2021 22:33:59 -0700 [thread overview]
Message-ID: <20210730053413.1090371-1-andrii@kernel.org> (raw)
This patch set implements an ability for users to specify custom black box u64
value for each BPF program attachment, bpf_cookie, which is available to BPF
program at runtime. This is a feature that's critically missing for cases when
some sort of generic processing needs to be done by the common BPF program
logic (or even exactly the same BPF program) across multiple BPF hooks (e.g.,
many uniformly handled kprobes) and it's important to be able to distinguish
between each BPF hook at runtime (e.g., for additional configuration lookup).
Currently, something like that can be only achieved through:
- code-generation and BPF program cloning, which is very complicated and
unmaintainable;
- on-the-fly C code generation and further runtime compilation, which is
what BCC uses and allows to do pretty simply. The big downside is a very
heavy-weight Clang/LLVM dependency and inefficient memory usage (due to
many BPF program clones and the compilation process itself);
- in some cases (kprobes and sometimes uprobes) it's possible to do function
IP lookup to get function-specific configuration. This doesn't work for
all the cases (e.g., when attaching uprobes to shared libraries) and has
higher runtime overhead and additional programming complexity due to
BPF_MAP_TYPE_HASHMAP lookups. Up until recently, before bpf_get_func_ip()
BPF helper was added, it was also very complicated and unstable (API-wise)
to get traced function's IP from fentry/fexit and kretprobe.
With libbpf and BPF CO-RE, runtime compilation is not an option, so to be able
to build generic tracing tooling simply and efficiently, ability to provide
additional bpf_cookie value for each *attachment* (as opposed to each BPF
program) is extremely important. Two immediate users of this functionality are
going to be libbpf-based USDT library (currently in development) and retsnoop
([0]), but I'm sure more applications will come once users get this feature in
their kernels.
To achieve above described, all perf_event-based BPF hooks are made available
through a new BPF_LINK_TYPE_PERF_EVENT BPF link, which allows to use common
LINK_CREATE command for program attachments and generally brings
perf_event-based attachments into a common BPF link infrastructure.
With that, LINK_CREATE gets ability to pass throught bpf_cookie value during
link creation (BPF program attachment) time. bpf_get_attach_cookie() BPF
helper is added to allow fetching this value at runtime from BPF program side.
BPF cookie is stored either on struct perf_event itself and fetched from the
BPF program context, or is passed through ambient BPF run context, added in
c7603cfa04e7 ("bpf: Add ambient BPF runtime context stored in current").
On the libbpf side of things, BPF perf link is utilized whenever is supported
by the kernel instead of using PERF_EVENT_IOC_SET_BPF ioctl on perf_event FD.
All the tracing attach APIs are extended with OPTS and bpf_cookie is passed
through corresponding opts structs.
Last part of the patch set adds few self-tests utilizing new APIs.
There are also a few refactorings along the way to make things cleaner and
easier to work with, both in kernel (BPF_PROG_RUN and BPF_PROG_RUN_ARRAY), and
throughout libbpf and selftests.
Follow-up patches will extend bpf_cookie to fentry/fexit programs.
[0] https://github.com/anakryiko/retsnoop
Cc: Peter Zijlstra <peterz@infradead.org> # for perf_event changes
v2->v3:
- user_ctx -> bpf_cookie, bpf_get_user_ctx -> bpf_get_attach_cookie (Peter);
- fix BPF_LINK_TYPE_PERF_EVENT value fix (Jiri);
- use bpf_prog_run() from bpf_prog_run_pin_on_cpu() (Yonghong);
v1->v2:
- fix build failures on non-x86 arches by gating on CONFIG_PERF_EVENTS.
Andrii Nakryiko (14):
bpf: refactor BPF_PROG_RUN into a function
bpf: refactor BPF_PROG_RUN_ARRAY family of macros into functions
bpf: refactor perf_event_set_bpf_prog() to use struct bpf_prog input
bpf: implement minimal BPF perf link
bpf: allow to specify user-provided bpf_cookie for BPF perf links
bpf: add bpf_get_attach_cookie() BPF helper to access bpf_cookie value
libbpf: re-build libbpf.so when libbpf.map changes
libbpf: remove unused bpf_link's destroy operation, but add dealloc
libbpf: use BPF perf link when supported by kernel
libbpf: add bpf_cookie support to bpf_link_create() API
libbpf: add bpf_cookie to perf_event, kprobe, uprobe, and tp attach
APIs
selftests/bpf: test low-level perf BPF link API
selftests/bpf: extract uprobe-related helpers into trace_helpers.{c,h}
selftests/bpf: add bpf_cookie selftests for high-level APIs
drivers/media/rc/bpf-lirc.c | 4 +-
include/linux/bpf.h | 206 ++++++++------
include/linux/bpf_types.h | 3 +
include/linux/filter.h | 63 +++--
include/linux/perf_event.h | 1 +
include/linux/trace_events.h | 7 +-
include/uapi/linux/bpf.h | 25 ++
kernel/bpf/cgroup.c | 32 +--
kernel/bpf/core.c | 29 +-
kernel/bpf/syscall.c | 105 +++++++-
kernel/events/core.c | 72 ++---
kernel/trace/bpf_trace.c | 45 +++-
tools/include/uapi/linux/bpf.h | 25 ++
tools/lib/bpf/Makefile | 10 +-
tools/lib/bpf/bpf.c | 32 ++-
tools/lib/bpf/bpf.h | 8 +-
tools/lib/bpf/libbpf.c | 196 +++++++++++---
tools/lib/bpf/libbpf.h | 71 ++++-
tools/lib/bpf/libbpf.map | 3 +
tools/lib/bpf/libbpf_internal.h | 32 ++-
.../selftests/bpf/prog_tests/attach_probe.c | 61 +----
.../selftests/bpf/prog_tests/bpf_cookie.c | 254 ++++++++++++++++++
.../selftests/bpf/prog_tests/perf_link.c | 89 ++++++
.../selftests/bpf/progs/test_bpf_cookie.c | 85 ++++++
.../selftests/bpf/progs/test_perf_link.c | 16 ++
tools/testing/selftests/bpf/trace_helpers.c | 66 +++++
tools/testing/selftests/bpf/trace_helpers.h | 3 +
27 files changed, 1230 insertions(+), 313 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_cookie.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/perf_link.c
create mode 100644 tools/testing/selftests/bpf/progs/test_bpf_cookie.c
create mode 100644 tools/testing/selftests/bpf/progs/test_perf_link.c
--
2.30.2
next reply other threads:[~2021-07-30 5:34 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-30 5:33 Andrii Nakryiko [this message]
2021-07-30 5:34 ` [PATCH v3 bpf-next 01/14] bpf: refactor BPF_PROG_RUN into a function Andrii Nakryiko
2021-07-30 17:19 ` Yonghong Song
2021-08-09 22:43 ` Daniel Borkmann
2021-08-10 0:28 ` Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 02/14] bpf: refactor BPF_PROG_RUN_ARRAY family of macros into functions Andrii Nakryiko
2021-08-09 23:00 ` Daniel Borkmann
2021-08-10 0:36 ` Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 03/14] bpf: refactor perf_event_set_bpf_prog() to use struct bpf_prog input Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 04/14] bpf: implement minimal BPF perf link Andrii Nakryiko
2021-07-30 17:24 ` Yonghong Song
2021-07-30 5:34 ` [PATCH v3 bpf-next 05/14] bpf: allow to specify user-provided bpf_cookie for BPF perf links Andrii Nakryiko
2021-07-30 22:02 ` Yonghong Song
2021-08-09 23:30 ` Daniel Borkmann
2021-08-10 0:52 ` Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 06/14] bpf: add bpf_get_attach_cookie() BPF helper to access bpf_cookie value Andrii Nakryiko
2021-07-30 22:13 ` Yonghong Song
2021-07-30 5:34 ` [PATCH v3 bpf-next 07/14] libbpf: re-build libbpf.so when libbpf.map changes Andrii Nakryiko
2021-07-30 22:31 ` Yonghong Song
2021-07-30 5:34 ` [PATCH v3 bpf-next 08/14] libbpf: remove unused bpf_link's destroy operation, but add dealloc Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 09/14] libbpf: use BPF perf link when supported by kernel Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 10/14] libbpf: add bpf_cookie support to bpf_link_create() API Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 11/14] libbpf: add bpf_cookie to perf_event, kprobe, uprobe, and tp attach APIs Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 12/14] selftests/bpf: test low-level perf BPF link API Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 13/14] selftests/bpf: extract uprobe-related helpers into trace_helpers.{c,h} Andrii Nakryiko
2021-07-30 5:34 ` [PATCH v3 bpf-next 14/14] selftests/bpf: add bpf_cookie selftests for high-level APIs Andrii Nakryiko
2021-08-01 5:11 ` [PATCH bpf-next v4] libbpf: introduce legacy kprobe events support Rafael David Tinoco
2021-08-06 22:29 ` Andrii Nakryiko
2021-09-12 6:48 ` [PATCH bpf-next v5] " Rafael David Tinoco
2021-09-12 6:52 ` Rafael David Tinoco
2021-09-14 4:44 ` Andrii Nakryiko
2021-09-14 5:02 ` Rafael David Tinoco
2021-09-14 5:21 ` Andrii Nakryiko
2021-09-14 14:27 ` sunyucong
2021-09-14 15:42 ` Rafael David Tinoco
2021-09-14 20:34 ` [PATCH bpf-next] libbpf: fix build error introduced by legacy kprobe feature Rafael David Tinoco
2021-09-14 21:04 ` Alexei Starovoitov
2021-09-14 21:35 ` [PATCH bpf-next v2] " Rafael David Tinoco
2021-09-14 21:39 ` Rafael David Tinoco
2021-09-14 21:48 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210730053413.1090371-1-andrii@kernel.org \
--to=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=kernel-team@fb.com \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.