[PATCH bpf-next 00/16] Add libbpf full support for BPF-to-BPF calls

* [PATCH bpf-next 00/16] Add libbpf full support for BPF-to-BPF calls
@ 2020-08-20 23:12 Andrii Nakryiko
  2020-08-20 23:12 ` [PATCH bpf-next 01/16] selftests/bpf: BPF object files should depend only on libbpf headers Andrii Nakryiko
                   ` (16 more replies)
  0 siblings, 17 replies; 21+ messages in thread
From: Andrii Nakryiko @ 2020-08-20 23:12 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

Currently, libbpf supports a limited form of BPF-to-BPF subprogram calls. The
restriction is that entry-point BPF program should use *all* of defined
sub-programs in BPF .o file. If any of the subprograms is not used, such
entry-point BPF program will be rejected by verifier as containing unreachable
dead code. This is not a big limitation for cases with single entry-point BPF
programs, but is quite a havy restriction for multi-programs that use only
partially overlapping set of subprograms.

This patch sets removes all such restrictions and adds complete support for
using BPF sub-program calls on BPF side. This is achieved through libbpf
tracking subprograms individually and detecting which subprograms are used by
any given entry-point BPF program, and subsequently only appending and
relocating code for just those used subprograms.

In addition, libbpf now also supports multiple entry-point BPF programs within
the same ELF section. This allows to structure code so that there are few
variants of BPF programs of the same type and attaching to the same target
(e.g., for tracepoints and kprobes) without the need to worry about ELF
section name clashes.

This patch set opens way for more wider adoption of BPF subprogram calls,
especially for real-world production use-cases with complicated net of
subprograms. This will allow to further scale BPF verification process through
good use of global functions, which can be verified independently. This is
also important prerequisite for static linking which allows static BPF
libraries to not worry about naming clashes for section names, as well as use
static non-inlined functions (subprograms) without worries of verifier
rejecting program due to dead code.

Patch set is structured as follows:
- patches 1-5 contain various smaller improvements to logging and selftests;
- patched 6-11 contain all the libbpf changes necessary to support multi-prog
  sections and bpf2bpf subcalls;
- patch 12 adds dedicated selftests validating all combinations of possible
  sub-calls (within and across sections, static vs global functions);
- patch 13 deprecated bpf_program__title() in favor of
  bpf_program__section_name(). The intent was to also deprecate
  bpf_object__find_program_by_title() as it's now non-sensical with multiple
  programs per section. But there were too many selftests uses of this and
  I didn't want to delay this patches further and make it even bigger, so left
  it for a follow up cleanup;
- patches 14-15 remove uses for title-related APIs from bpftool and
  bpf_program__title() use from selftests;
- patch 16 is converting fexit_bpf2bpf to have explicit subtest (it does
  contain 4 subtests, which are not handled as sub-tests).

Andrii Nakryiko (16):
  selftests/bpf: BPF object files should depend only on libbpf headers
  libbpf: factor out common ELF operations and improve logging
  libbpf: add __noinline macro to bpf_helpers.h
  libbpf: skip well-known ELF sections when iterating ELF
  libbpf: normalize and improve logging across few functions
  libbpf: ensure ELF symbols table is found before further ELF
    processing
  libbpf: parse multi-function sections into multiple BPF programs
  libbpf: support CO-RE relocations for multi-prog sections
  libbpf: make RELO_CALL work for multi-prog sections and sub-program
    calls
  libbpf: implement generalized .BTF.ext func/line info adjustment
  libbpf: add multi-prog section support for struct_ops
  selftests/bpf: add selftest for multi-prog sections and bpf-to-bpf
    calls
  tools/bpftool: replace bpf_program__title() with
    bpf_program__section_name()
  selftests/bpf: don't use deprecated libbpf APIs
  libbpf: deprecate notion of BPF program "title" in favor of "section
    name"
  selftests/bpf: turn fexit_bpf2bpf into test with subtests

 tools/bpf/bpftool/prog.c                      |    4 +-
 tools/lib/bpf/bpf_helpers.h                   |    3 +
 tools/lib/bpf/btf.h                           |   18 +-
 tools/lib/bpf/libbpf.c                        | 1695 ++++++++++-------
 tools/lib/bpf/libbpf.h                        |    5 +-
 tools/lib/bpf/libbpf.map                      |    5 +
 tools/lib/bpf/libbpf_common.h                 |    2 +
 tools/testing/selftests/bpf/Makefile          |    2 +-
 .../selftests/bpf/flow_dissector_load.h       |    8 +-
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |   12 +-
 .../bpf/prog_tests/reference_tracking.c       |    2 +-
 .../selftests/bpf/prog_tests/subprogs.c       |   31 +
 .../selftests/bpf/progs/test_subprogs.c       |   92 +
 .../selftests/bpf/test_socket_cookie.c        |    2 +-
 14 files changed, 1218 insertions(+), 663 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/subprogs.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_subprogs.c

-- 
2.24.1

^ permalink raw reply	[flat|nested] 21+ messages in thread