Re: libbpf/BTF loading issue with fentry/fexit selftests

[Matt Cover <werekraken@gmail.com>]

On Wed, Mar 25, 2020 at 6:31 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Wed, Mar 25, 2020 at 3:16 PM Matt Cover <werekraken@gmail.com> wrote:
> >
> > I'm looking to explore the bpf trampoline Alexei introduced for
> > tracing progs, but am encountering a libbpf/BTF issue with loading
> > the selftests. Hoping you guys might have a pointer or two.
> >
> > The kernel build used pahole 1.15. All llvm-project components used
> > in compiling the selftests were 10.0.0-rc6.
> >
> > I believe the following confirms that BTF is indeed present in this kernel.
>
> BTF is, but that BTF doesn't have information about FUNCs (only
> FUNC_PROTOs). You need pahole 1.16 for fentry/fexit.

Thanks Andrii! pahole 1.16 fixed things up.

[vagrant@localhost bpf]$ uname -r
5.5.9-1.btf.2.el7.x86_64
[vagrant@localhost bpf]$ ~/bpftool btf dump file
/sys/kernel/btf/vmlinux | grep -i 'fentry\|fexit'
    'BPF_TRAMP_FENTRY' val=0
    'BPF_TRAMP_FEXIT' val=1
    'BPF_TRACE_FENTRY' val=24
    'BPF_TRACE_FEXIT' val=25
    'fentry_progs' type_id=1950
    'fentry_cnt' type_id=18
    'fexit_progs' type_id=1950
    'fexit_cnt' type_id=18
[59594] FUNC 'bpf_fentry_test1' type_id=59593
[59596] FUNC 'bpf_fentry_test2' type_id=59595
[59598] FUNC 'bpf_fentry_test3' type_id=59597
[59600] FUNC 'bpf_fentry_test4' type_id=59599
[59602] FUNC 'bpf_fentry_test5' type_id=59601
[59604] FUNC 'bpf_fentry_test6' type_id=59603
[vagrant@localhost bpf]$ sudo ./test_progs -t fexit_test
#10 fexit_test:OK
Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

>
> >
> >
> > [vagrant@localhost bpf]$ uname -r
> > 5.5.9-1.btf.el7.x86_64
> > [vagrant@localhost bpf]$ grep CONFIG_DEBUG_INFO_BTF /boot/config-`uname -r`
> > CONFIG_DEBUG_INFO_BTF=y
> > [vagrant@localhost bpf]$ ~/bpftool btf dump file ~/vmlinux-`uname -r`
> > | grep -i fexit
> >     'BPF_TRAMP_FEXIT' val=1
> >     'BPF_TRACE_FEXIT' val=25
> > [vagrant@localhost bpf]$ ~/bpftool btf dump file
> > /sys/kernel/btf/vmlinux | grep -i fexit
> >     'BPF_TRAMP_FEXIT' val=1
> >     'BPF_TRACE_FEXIT' val=25
> >
> >
> > The fexit_test.o file also has BTF information.
> >
> >
> > [vagrant@localhost bpf]$ ~/bpftool btf dump file fexit_test.o | grep FUNC_PROTO
> > [4] FUNC_PROTO '(anon)' ret_type_id=5 vlen=1
> > [7] FUNC_PROTO '(anon)' ret_type_id=5 vlen=1
> > [9] FUNC_PROTO '(anon)' ret_type_id=5 vlen=1
> > [11] FUNC_PROTO '(anon)' ret_type_id=5 vlen=1
> > [13] FUNC_PROTO '(anon)' ret_type_id=5 vlen=1
> > [15] FUNC_PROTO '(anon)' ret_type_id=5 vlen=1
> >
> >
> > However, I get libbpf/BTF load errors when trying to run any
> > fentry/fexit tests.
> >
> >
> > [vagrant@localhost bpf]$ sudo ./test_progs -t fexit_test | grep '^libbpf\|FAIL'
> > libbpf: Error loading BTF: Invalid argument(22)
> > libbpf: magic: 0xeb9f
> > libbpf: Error loading .BTF into kernel: -22.
> > libbpf: Error loading BTF: Invalid argument(22)
> > libbpf: magic: 0xeb9f
> > libbpf: Error loading .BTF into kernel: -22.
> > libbpf: fexit/bpf_fentry_test1 is not found in vmlinux BTF
> > test_fexit_test:FAIL:prog_load fail err -2 errno 22
> > #10 fexit_test:FAIL
> > Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED
> >
> >
> > I saw in a similar thread that -vvv output was requested. Figured the
> > same applies here.
>
> Yeah, for tricky issues that good. In this case it was pretty obvious,
> but generally it's a good idea for sure, thanks!
> >
> >
> > [vagrant@localhost bpf]$ sudo ./test_progs -vvv -t fexit_test | grep
> > '^libbpf\|FAIL'
>
> [...]
>
> >
> >
> > Any hints on the issue?
> >
> > -Matt C.