Re: Where can I find the map's BTF type key/value specification?

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Sat, Oct 10, 2020 at 3:50 AM Daniel T. Lee <danieltimlee@gmail.com> wrote:
>
> I'm looking for how BTF type definition '__type(key, int)' is being changed
> to '__uint(key_size, sizeof(int))'. (Not exactly "changed" but wonder how
> it can be considered the same)

__type(key, int) captures both BTF ID of key and determines key_size
based on that type. You can specify both key and key_size, but that's
unnecessary (and resulting key size still has to match).

>
>     __uint(type, BPF_MAP_TYPE_ARRAY);
>     __type(key, int);          => __uint(key_size, sizeof(int))
>     __type(value, u32);    => __uint(value_size, sizeof(u32))
>     __uint(max_entries, 2);
>
> Whether the specific map type supports BTF or not can be inferred from
> the file in kernel/bpf/*map.c and by checking each MAP type's
> bpf_map_ops .map_check_btf pointer is initialized as map_check_no_btf.
>
> But how can I figure out that specific types of map support BTF types for
> key/value? And how can I determine how this BTF key/value type is
> converted?

I think you answered your own question, you just search whether each
map implements .map_check_btf that allows key/value BTF type ID. E.g.,
see array_map_check_btf, which allows key/value type ID. And compare
to how perf_event_array_map_ops use map_check_no_btf for its
.map_check_btf callback.

So you can search for all struct bpf_map_ops declarations to see
operations for all map types, and then see what's there for
.map_check_btf. Ideally we should extend all maps to support BTF type
ID for key/value, but no one signed up to do that. If you are
interested, that should be a good way to contribute to kernel itself.

>
> I am aware that BTF information is created in the form of a compact
> type by using pahole to deduplicate repeated types, strings information
> from DWARF information. However, looking at the *btf or pahole file
> in dwarves repository, it seemed that it was not responsible for the
> conversion of the BTF key/value.
>
> The remaining guess is that LLVM's BPF target compiler is responsible
> for this, or it's probably somewhere in the kernel, but I'm not sure
> where it is.

BTF for the BPF program is emitted by Clang itself when you specify
`-target bpf -g`. pahole is used to convert kernel's (vmlinux) DWARF
into BTF and embed it into vmlinux image.

As for key/value BTF type id for maps, that's libbpf parsing map
definition and recording type IDs. So there are a few things playing
together. See abd29c931459 ("libbpf: allow specifying map definitions
using BTF") that introduced this feature.

>
> --
> Best,
> Daniel T. Lee