Re: pahole generates invalid BTF for code compiled with recent clang

[Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

On Thu, Jun 25, 2020 at 2:25 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
>
> On Wed, 24 Jun 2020 at 18:41, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > >
> > > Hi,
> > >
> > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > produces an invalid
> > > output. This is because pahole rewrites the .BTF section (which
> > > includes a new string
> > > table) but it doesn't touch .BTF.ext at all.
> >
> > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > .BTF (and .BTF.ext, of course) for you.
>
> You're missing the point. The kernel build system does it. Try the following:

Yeah, I clearly am, because "compiling old kernels like 4.19" made me
think that we are talking about building kernel, not selftests.

>
> * Get the v4.19 sources
> * Make sure that clang --version is 10
> * Make sure you have pahole (I used v1.17)
> * Build selftests
>
> The resulting object files will have bogus .BTF.ext sections due the
> bug I have described. Does it make sense to run pahole -J on these?
> No, but it still happens.
>

Yeah, because back in the day Clang didn't know how to generate .BTF,
so using pahole to generate .BTF for BPF object files was a solution.

> I think it's reasonable to expect to get valid BPF ELFs out of this process.

We should probably update Makefile for old kernel selftest to not call
pahole -J, if Clang is recent enough and/or if .o file already has
.BTF. That shouldn't be hard.

>
> >
> > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > -target BPF object files.
> >
> > >
> > > To demonstrate, on a recent check out of bpf-next:
> > >     $ cp connect4_prog.o connect4_pahole.o
> > >     $ pahole -J connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > .BTF.ext=btf-ext.bin connect4_prog.o
> > >     $ sha1sum *.bin
> > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > >
> > > This problem crops up when compiling old kernels like 4.19 which have
> > > an extra pahole
> > > build step with clang-10.
> >
> > I was under impression that clang generates .BTF and .BTF.ext only for
> > -target BPF. In this case, kernel is compiled for "real" target arch,
> > so there shouldn't be .BTF.ext in the first place? If that's not the
> > case, then I guess it's a bug in Clang.
>
> connect4_prog.o is BPF:
>
> $ readelf -h connect4_prog.o | grep BPF
>   Machine:                           Linux BPF
>
> Maybe I misunderstand what you're trying to say.
>

I was talking/thinking about building kernel, you were talking about
building BPF object files in selftests. Just to avoid confusion in the
future, let's not talk about compiling kernel, when we are talking
about compiling selftests. See my suggestion for a fix above. Would
that work?

> Best
> Lorenz
>
> --
> Lorenz Bauer  |  Systems Engineer
> 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
>
> www.cloudflare.com