[PATCH v3 bpf-next 0/4] Add libbpf-provided extern variables support

* [PATCH v3 bpf-next 0/4] Add libbpf-provided extern variables support
@ 2019-12-13 23:51 Andrii Nakryiko
  2019-12-13 23:51 ` [PATCH v3 bpf-next 1/4] libbpf: extract internal map names into constants Andrii Nakryiko
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: Andrii Nakryiko @ 2019-12-13 23:51 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

It's often important for BPF program to know kernel version or some specific
config values (e.g., CONFIG_HZ to convert jiffies to seconds) and change or
adjust program logic based on their values. As of today, any such need has to
be resolved by recompiling BPF program for specific kernel and kernel
configuration. In practice this is usually achieved by using BCC and its
embedded LLVM/Clang. With such set up #ifdef CONFIG_XXX and similar
compile-time constructs allow to deal with kernel varieties.

With CO-RE (Compile Once – Run Everywhere) approach, this is not an option,
unfortunately. All such logic variations have to be done as a normal
C language constructs (i.e., if/else, variables, etc), not a preprocessor
directives. This patch series add support for such advanced scenarios through
C extern variables. These extern variables will be recognized by libbpf and
supplied through extra .extern internal map, similarly to global data. This
.extern map is read-only, which allows BPF verifier to track its content
precisely as constants. That gives an opportunity to have pre-compiled BPF
program, which can potentially use BPF functionality (e.g., BPF helpers) or
kernel features (types, fields, etc), that are available only on a subset of
targeted kernels, while effectively eleminating (through verifier's dead code
detection) such unsupported functionality for other kernels (typically, older
versions). Patch #3 explicitly tests a scenario of using unsupported BPF
helper, to validate the approach.

This patch set heavily relies on BTF type information emitted by compiler for
each extern variable declaration. Based on specific types, libbpf does strict
checks of config data values correctness. See patch #1 for details.

Outline of the patch set:
- patch #1 does a small clean up of internal map names contants;
- patch #2 adds all of the libbpf internal machinery for externs support,
  including setting up BTF information for .extern data section;
- patch #3 adds support for .extern into BPF skeleton;
- patch #4 adds externs selftests, as well as enhances test_skeleton.c test to
  validate mmap()-ed .extern datasection functionality.

This patch set is based off BPF skeleton patch set ([0]).

  [0] https://patchwork.ozlabs.org/project/netdev/list/?series=148468&state=*

v2->v3:
- truncate too long strings (Alexei);
- clean ups, adding comments (Alexei);

v1->v2:
- use BTF type information for externs (Alexei);
- add strings support;
- add BPF skeleton support for .extern.

Andrii Nakryiko (4):
  libbpf: extract internal map names into constants
  libbpf: support libbpf-provided extern variables
  bpftool: generate externs datasec in BPF skeleton
  selftests/bpf: add tests for libbpf-provided externs

 include/uapi/linux/btf.h                      |   3 +-
 tools/bpf/bpftool/gen.c                       |   4 +
 tools/include/uapi/linux/btf.h                |   7 +-
 tools/lib/bpf/Makefile                        |  15 +-
 tools/lib/bpf/bpf_helpers.h                   |   9 +
 tools/lib/bpf/btf.c                           |   9 +-
 tools/lib/bpf/libbpf.c                        | 786 ++++++++++++++++--
 tools/lib/bpf/libbpf.h                        |  12 +-
 tools/testing/selftests/bpf/Makefile          |   2 +-
 .../selftests/bpf/prog_tests/core_extern.c    | 193 +++++
 .../selftests/bpf/prog_tests/skeleton.c       |  18 +-
 .../selftests/bpf/progs/test_core_extern.c    |  62 ++
 .../selftests/bpf/progs/test_skeleton.c       |   9 +
 13 files changed, 1042 insertions(+), 87 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/core_extern.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_core_extern.c

-- 
2.17.1

^ permalink raw reply	[flat|nested] 8+ messages in thread