[PATCH bpf-next v2 0/4] libbpf: Resolve unambigous forward declarations

* [PATCH bpf-next v2 0/4] libbpf: Resolve unambigous forward declarations
@ 2022-11-03  3:34 Eduard Zingerman
  2022-11-03  3:34 ` [PATCH bpf-next v2 1/4] libbpf: hashmap interface update to uintptr_t -> uintptr_t Eduard Zingerman
                   ` (3 more replies)
  0 siblings, 4 replies; 10+ messages in thread
From: Eduard Zingerman @ 2022-11-03  3:34 UTC (permalink / raw)
  To: bpf, ast; +Cc: andrii, daniel, kernel-team, yhs, alan.maguire, Eduard Zingerman

Resolve forward declarations that don't take part in type graphs
comparisons if declaration name is unambiguous.
Example:

CU #1:

struct foo;              // standalone forward declaration
struct foo *some_global;

CU #2:

struct foo { int x; };
struct foo *another_global;

Currently the de-duplicated BTF for this example looks as follows:

[1] STRUCT 'foo' size=4 vlen=1 ...
[2] INT 'int' size=4 ...
[3] PTR '(anon)' type_id=1
[4] FWD 'foo' fwd_kind=struct
[5] PTR '(anon)' type_id=4

The goal of this patch-set is to simplify it as follows:

[1] STRUCT 'foo' size=4 vlen=1
	'x' type_id=2 bits_offset=0
[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] PTR '(anon)' type_id=1

The patch-set is consists of the following parts:
- A refactoring of the libbpf's hashmap interface to use `uintptr_t`
  instead of `void*` for keys and values. The reasoning behind the
  refactoring is that integer keys / values are used in libbpf more
  often then pointer keys / values. Thus the refactoring reduces the
  number of awkward looking casts like "(void *)(long)off".
  `uintptr_t` is used to avoid necessity for temporary variables when
  pointer keys / values are used on platforms with 32-bit pointers.
- A change to `lib/bpf/btf.c:btf__dedup` that adds a new pass named
  "Resolve unambiguous forward declaration". This pass builds a
  hashmap `name_off -> uniquely named struct or union` and uses it to
  replace FWD types by structs or unions. This is necessary for corner
  cases when FWD is not used as a part of some struct or union
  definition de-duplicated by `btf_dedup_struct_types`.

For defconfig kernel with BTF enabled this removes 63 forward
declarations.

For allmodconfig kernel with BTF enabled this removes ~5K out of ~21K
forward declarations in ko objects. This unlocks some additional
de-duplication in ko objects, but impact is tiny: ~13K less BTF ids
out of ~2M.

Changelog:
 v1 -> v2
 v1: https://lore.kernel.org/bpf/20221102110905.2433622-1-eddyz87@gmail.com/T/#t
 - Style fixes in btf_dedup_resolve_fwd and btf_dedup_resolve_fwds as
   suggested by Alan.

Eduard Zingerman (4):
  libbpf: hashmap interface update to uintptr_t -> uintptr_t
  selftests/bpf: hashmap test cases updated for uintptr_t -> uintptr_t
    interface
  libbpf: Resolve unambigous forward declarations
  selftests/bpf: Tests for btf_dedup_resolve_fwds

 tools/bpf/bpftool/btf.c                       |  23 +--
 tools/bpf/bpftool/common.c                    |  10 +-
 tools/bpf/bpftool/gen.c                       |  19 +-
 tools/bpf/bpftool/link.c                      |   8 +-
 tools/bpf/bpftool/main.h                      |   4 +-
 tools/bpf/bpftool/map.c                       |   8 +-
 tools/bpf/bpftool/pids.c                      |  16 +-
 tools/bpf/bpftool/prog.c                      |   8 +-
 tools/lib/bpf/btf.c                           | 186 +++++++++++++++---
 tools/lib/bpf/btf_dump.c                      |  16 +-
 tools/lib/bpf/hashmap.c                       |  16 +-
 tools/lib/bpf/hashmap.h                       |  35 ++--
 tools/lib/bpf/libbpf.c                        |  18 +-
 tools/lib/bpf/strset.c                        |  24 +--
 tools/lib/bpf/usdt.c                          |  31 ++-
 tools/testing/selftests/bpf/prog_tests/btf.c  | 152 ++++++++++++++
 .../bpf/prog_tests/btf_dedup_split.c          |  45 +++--
 .../selftests/bpf/prog_tests/hashmap.c        |  68 +++----
 .../bpf/prog_tests/kprobe_multi_test.c        |   6 +-
 19 files changed, 485 insertions(+), 208 deletions(-)

-- 
2.34.1

^ permalink raw reply	[flat|nested] 10+ messages in thread