Re: [PATCH bpf-next 1/7] libbpf: add BPF-side of USDT support

From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Hengqi Chen <hengqi.chen@gmail.com>
Cc: Andrii Nakryiko <andrii@kernel.org>, bpf <bpf@vger.kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Kernel Team <kernel-team@fb.com>,
	Alan Maguire <alan.maguire@oracle.com>,
	Dave Marchevsky <davemarchevsky@fb.com>
Subject: Re: [PATCH bpf-next 1/7] libbpf: add BPF-side of USDT support
Date: Wed, 30 Mar 2022 22:48:14 -0700	[thread overview]
Message-ID: <CAEf4Bza-=c=d-aDz0Ruv2AZrn5fh+5HqL-gvUWa7c9=o+PuWYA@mail.gmail.com> (raw)
In-Reply-To: <605dc1f0-2c66-25f0-ef76-a3c052fcc2d8@gmail.com>

On Wed, Mar 30, 2022 at 8:36 AM Hengqi Chen <hengqi.chen@gmail.com> wrote:
>
>
>
> On 2022/3/30 11:10 AM, Hengqi Chen wrote:
> > On 2022/3/25 1:29 PM, Andrii Nakryiko wrote:
> >> Add BPF-side implementation of libbpf-provided USDT support. This
> >> consists of single header library, usdt.bpf.h, which is meant to be used
> >> from user's BPF-side source code. This header is added to the list of
> >> installed libbpf header, along bpf_helpers.h and others.
> >>
> >> BPF-side implementation consists of two BPF maps:
> >>   - spec map, which contains "a USDT spec" which encodes information
> >>     necessary to be able to fetch USDT arguments and other information
> >>     (argument count, user-provided cookie value, etc) at runtime;
> >>   - IP-to-spec-ID map, which is only used on kernels that don't support
> >>     BPF cookie feature. It allows to lookup spec ID based on the place
> >>     in user application that triggers USDT program.
> >>
> >> These maps have default sizes, 256 and 1024, which are chosen
> >> conservatively to not waste a lot of space, but handling a lot of common
> >> cases. But there could be cases when user application needs to either
> >> trace a lot of different USDTs, or USDTs are heavily inlined and their
> >> arguments are located in a lot of differing locations. For such cases it
> >> might be necessary to size those maps up, which libbpf allows to do by
> >> overriding BPF_USDT_MAX_SPEC_CNT and BPF_USDT_MAX_IP_CNT macros.
>
> >> +
> >> +__weak struct {
> >> +    __uint(type, BPF_MAP_TYPE_ARRAY);
> >> +    __uint(max_entries, BPF_USDT_MAX_SPEC_CNT);
> >> +    __type(key, int);
> >> +    __type(value, struct __bpf_usdt_spec);
> >> +} __bpf_usdt_specs SEC(".maps");
> >> +
> >> +__weak struct {
> >> +    __uint(type, BPF_MAP_TYPE_HASH);
> >> +    __uint(max_entries, BPF_USDT_MAX_IP_CNT);
> >> +    __type(key, long);
> >> +    __type(value, struct __bpf_usdt_spec);
> >
> > type should be int.
> >
> >> +} __bpf_usdt_specs_ip_to_id SEC(".maps");
>
> These weak symbols make BPF object open failed:
>
> libbpf: No offset found in symbol table for VAR __bpf_usdt_specs
> libbpf: Error finalizing .BTF: -2.
>
>     bpf_object_open
>         bpf_object__finalize_btf
>             btf_finalize_data
>                 btf_fixup_datasec
>                     find_elf_var_offset
>
> This is because during BTF fixup, we only allow GLOBAL VAR.
>
> Applying the following diff can workaround the issue:
>
> +               unsigned char bind = ELF64_ST_BIND(sym->st_info);
>
> -               if (ELF64_ST_BIND(sym->st_info) != STB_GLOBAL ||
> +               if ((bind != STB_GLOBAL && bind != STB_WEAK) ||
>
>

Interesting that selftests don't run into this bug, probably because
BPF linker converts STB_WEAK into STB_GLOBAL? I'll check that, thanks
for catching!

> >> +#endif /* __USDT_BPF_H__ */