bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Mauricio Vásquez" <mauricio@kinvolk.io>
To: netdev@vger.kernel.org, bpf@vger.kernel.org
Cc: Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Rafael David Tinoco <rafaeldtinoco@gmail.com>
Subject: [PATCH bpf-next 0/2] libbpf: Implement BTF Generator API
Date: Wed, 27 Oct 2021 15:37:25 -0500	[thread overview]
Message-ID: <20211027203727.208847-1-mauricio@kinvolk.io> (raw)

CO-RE requires to have BTF information describing the types of the
kernel in order to perform the relocations. This is usually provided by
the kernel itself when it's configured with CONFIG_DEBUG_INFO_BTF.
However, this configuration is not enabled in all the distributions and
it's not available on older kernels.

It's possible to use CO-RE in kernels without CONFIG_DEBUG_INFO_BTF
support by providing the BTF information from an external source.
BTFHub[0] contains BTF files to each released kernel not supporting BTF,
for the most popular distributions.

Providing this BTF file for a given kernel has some challenges:
1. Each BTF file is a few MBs big, then it's not possible to ship the
eBPF program with the all the BTF files needed to run in different
kernels. (The BTF files will be in the order of GBs if you want to
support a high number of kernels)
2. Downloading the BTF file for the current kernel at runtime delays the
start of the program and it's not always possible to reach an external
host to download such a file.

Providing the BTF file with the information about all the data types of
the kernel for running an eBPF program is an overkill in many of the
cases. Usually the eBPF programs access only some kernel fields.

This set of commits extend libbpf to provide an API to generate a BTF
file with only the types that are needed by an eBPF object. These
generated files are very small compared to the ones that contain all the
kernel types (for a program like execsnoop it's around 4kB). This allows
to ship an eBPF program together with the BTF information that it needs
to run for many different kernels.

This idea was discussed during the "Towards truly portable eBPF"[1]
presentation at Linux Plumbers 2021.

We prepared a BTFGen repository[2] with an example of how this API can
be used. Our plan is to include this support in bpftool once it's merged
in libbpf.

There is also a good example[3] on how to use BTFGen and BTFHub together
to generate multiple BTF files, to each existing/supported kernel,
tailored to one application. For example: a complex bpf object might
support nearly 400 kernels by having BTF files summing only 1.5 MB.

[0]: https://github.com/aquasecurity/btfhub/
[1]: https://www.youtube.com/watch?v=igJLKyP1lFk&t=2418s
[2]: https://github.com/kinvolk/btfgen
[3]: https://github.com/aquasecurity/btfhub/tree/main/tools

Mauricio Vásquez (2):
  libbpf: Implement btf__save_to_file()
  libbpf: Implement API for generating BTF for ebpf objects

 tools/lib/bpf/Makefile    |   2 +-
 tools/lib/bpf/btf.c       |  22 ++
 tools/lib/bpf/btf.h       |   2 +
 tools/lib/bpf/libbpf.c    |  28 ++-
 tools/lib/bpf/libbpf.h    |   4 +
 tools/lib/bpf/libbpf.map  |   6 +
 tools/lib/bpf/relo_core.c | 514 +++++++++++++++++++++++++++++++++++++-
 tools/lib/bpf/relo_core.h |  11 +-
 8 files changed, 579 insertions(+), 10 deletions(-)


             reply	other threads:[~2021-10-27 20:37 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-27 20:37 Mauricio Vásquez [this message]
2021-10-27 20:37 ` [PATCH bpf-next 1/2] libbpf: Implement btf__save_to_file() Mauricio Vásquez
2021-10-28 18:36   ` Andrii Nakryiko
2021-10-27 20:37 ` [PATCH bpf-next 2/2] libbpf: Implement API for generating BTF for ebpf objects Mauricio Vásquez
2021-10-28 18:45   ` Andrii Nakryiko
2021-10-28 22:42     ` Mauricio Vásquez Bernal
2021-10-28 22:48       ` Andrii Nakryiko
2021-10-29  2:33 ` [PATCH bpf-next 0/2] libbpf: Implement BTF Generator API Alexei Starovoitov
2021-10-29  5:41   ` Rafael David Tinoco
2021-10-29  5:51     ` Rafael David Tinoco
2021-10-29 16:12   ` Mauricio Vásquez Bernal
2021-11-02  5:53     ` Andrii Nakryiko
2021-11-02 10:58       ` Leonardo Di Donato
2021-11-02 17:12         ` Andrii Nakryiko
2021-11-02 21:26       ` Mauricio Vásquez Bernal
2021-11-03  5:26         ` Andrii Nakryiko
2021-11-04 14:58           ` Mauricio Vásquez Bernal
2021-11-04 17:34             ` Andrii Nakryiko
2021-11-03 23:40       ` Andrii Nakryiko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211027203727.208847-1-mauricio@kinvolk.io \
    --to=mauricio@kinvolk.io \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=netdev@vger.kernel.org \
    --cc=rafaeldtinoco@gmail.com \
    --subject='Re: [PATCH bpf-next 0/2] libbpf: Implement BTF Generator API' \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).