[PATCH bpf-next 00/11] Future-proof more tricky libbpf APIs

* [PATCH bpf-next 00/11] Future-proof more tricky libbpf APIs
@ 2021-11-08  6:13 Andrii Nakryiko
  2021-11-08  6:13 ` [PATCH bpf-next 01/11] bpftool: normalize compile rules to specify output file last Andrii Nakryiko
                   ` (10 more replies)
  0 siblings, 11 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2021-11-08  6:13 UTC (permalink / raw)
  To: bpf, ast, daniel; +Cc: andrii, kernel-team

This patch set continues the work of revamping libbpf APIs that are not
extensible, as they were added before we figured out all the intricacies of
building APIs that can preserve ABI compatibility (both backward and forward).

What makes them tricky is that (most of) these APIs are actively used by
multiple applications, so we need to be careful about refactoring them. See
individual patches for details, but the general approach is similar to
previous bpf_prog_load() API revamp. The biggest different and complexity is
in changing btf_dump__new(), because function overloading through macro magic
doesn't work based on number of arguments, as both new and old APIs have
4 arguments. Because of that, another overloading approach is taken; overload
happens based on argument types.

To validate that not just source code compatibility is preserved, but that
applications using libbpf as a shared library stay compatible as well, first
few patches extend selftests/bpf with another flavor of test_progs (-shared),
that compiles against libbpf.so. Using this flavor I've validated manually
that compiling "old application" (selftests before being adapted to using new
variants of revamped APIs) are compiled and successfully run against newest
libbpf version as well as the older libbpf version (provided no new variants
are used). All these scenarios seem to be working as expected.

Currently, the way that selftests are set up we'll build three variants of BPF
object files (alu32, no-alu32, and alu32 but for shared flavor), which is
suboptimal. It is possible to rework Makefile to only generate alu32 vs
no-alu32 variants and use them in appropriate flavors, minimizing unnecessary
work. But that was left for follow up patches as there are already a lot of
changes across selftests and libbpf.

Andrii Nakryiko (11):
  bpftool: normalize compile rules to specify output file last
  selftests/bpf: minor cleanups and normalization of Makefile
  selftests/bpf: allow to generate per-flavor list of tests
  selftests/bpf: add test_progs flavor using libbpf as a shared lib
  libbpf: turn btf_dedup_opts into OPTS-based struct
  libbpf: ensure btf_dump__new() and btf_dump_opts are future-proof
  libbpf: make perf_buffer__new() use OPTS-based interface
  selftests/bpf: migrate all deprecated perf_buffer uses
  selftests/bpf: update btf_dump__new() uses to v1.0+ variant
  tools/runqslower: update perf_buffer__new() calls
  bpftool: update btf_dump__new() and perf_buffer__new_raw() calls

 tools/bpf/bpftool/Makefile                    | 16 ++--
 tools/bpf/bpftool/btf.c                       |  2 +-
 tools/bpf/bpftool/gen.c                       |  2 +-
 tools/bpf/bpftool/map_perf_ring.c             |  9 +-
 tools/bpf/runqslower/runqslower.c             |  6 +-
 tools/lib/bpf/btf.c                           | 46 +++++----
 tools/lib/bpf/btf.h                           | 72 ++++++++++++--
 tools/lib/bpf/btf_dump.c                      | 26 ++++--
 tools/lib/bpf/libbpf.c                        | 70 +++++++++++---
 tools/lib/bpf/libbpf.h                        | 86 ++++++++++++++---
 tools/lib/bpf/libbpf.map                      |  8 ++
 tools/lib/bpf/linker.c                        |  4 +-
 tools/testing/selftests/bpf/.gitignore        |  9 +-
 tools/testing/selftests/bpf/Makefile          | 93 +++++++++++--------
 .../selftests/bpf/benchs/bench_ringbufs.c     |  8 +-
 tools/testing/selftests/bpf/btf_helpers.c     |  4 +-
 tools/testing/selftests/bpf/prog_tests/btf.c  | 46 ++-------
 .../bpf/prog_tests/btf_dedup_split.c          |  6 +-
 .../selftests/bpf/prog_tests/btf_dump.c       | 33 +++----
 .../selftests/bpf/prog_tests/btf_split.c      |  4 +-
 .../bpf/prog_tests/get_stack_raw_tp.c         |  5 +-
 .../selftests/bpf/prog_tests/kfree_skb.c      |  6 +-
 .../selftests/bpf/prog_tests/perf_buffer.c    |  6 +-
 .../selftests/bpf/prog_tests/xdp_bpf2bpf.c    |  7 +-
 tools/testing/selftests/bpf/test_maps.c       |  4 +-
 tools/testing/selftests/bpf/test_progs.c      |  6 +-
 .../selftests/bpf/test_tcpnotify_user.c       |  4 +-
 27 files changed, 363 insertions(+), 225 deletions(-)

-- 
2.30.2

^ permalink raw reply	[flat|nested] 20+ messages in thread