pahole generates invalid BTF for code compiled with recent clang

pahole generates invalid BTF for code compiled with recent clang

[Lorenz Bauer]

Hi,

If pahole -J is used on an ELF that has BTF info from clang, it
produces an invalid
output. This is because pahole rewrites the .BTF section (which
includes a new string
table) but it doesn't touch .BTF.ext at all.

To demonstrate, on a recent check out of bpf-next:
    $ cp connect4_prog.o connect4_pahole.o
    $ pahole -J connect4_pahole.o
    $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
--dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
    $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
.BTF.ext=btf-ext.bin connect4_prog.o
    $ sha1sum *.bin
    1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
    4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
    2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
    4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin

This problem crops up when compiling old kernels like 4.19 which have
an extra pahole
build step with clang-10.

I think a possible fix is to strip .BTF.ext if .BTF is rewritten.

Best
Lorenz

-- 
Lorenz Bauer  |  Systems Engineer
6th Floor, County Hall/The Riverside Building, SE1 7PB, UK

www.cloudflare.com

Re: pahole generates invalid BTF for code compiled with recent clang

[Arnaldo Carvalho de Melo]

Em Wed, Jun 24, 2020 at 12:05:50PM +0100, Lorenz Bauer escreveu:
> Hi,
> 
> If pahole -J is used on an ELF that has BTF info from clang, it
> produces an invalid
> output. This is because pahole rewrites the .BTF section (which
> includes a new string
> table) but it doesn't touch .BTF.ext at all.
 
> To demonstrate, on a recent check out of bpf-next:
>     $ cp connect4_prog.o connect4_pahole.o
>     $ pahole -J connect4_pahole.o
>     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
>     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> .BTF.ext=btf-ext.bin connect4_prog.o
>     $ sha1sum *.bin
>     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
>     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
>     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
>     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> 
> This problem crops up when compiling old kernels like 4.19 which have
> an extra pahole
> build step with clang-10.

 
> I think a possible fix is to strip .BTF.ext if .BTF is rewritten.

Agreed.

Longer term pahole needs to generate the .BTF.ext from DWARF, but then,
if clang is generating it already, why use pahole -J?

Does clang do deduplication for multi-object binaries?

Also its nice to see that the BTF generated ends up with the same
sha1sum, cool :-)
 
> Best
> Lorenz
> 
> -- 
> Lorenz Bauer  |  Systems Engineer
> 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> 
> www.cloudflare.com

-- 

- Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Lorenz Bauer]

On Wed, 24 Jun 2020 at 17:07, Arnaldo Carvalho de Melo <acme-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>
> Em Wed, Jun 24, 2020 at 12:05:50PM +0100, Lorenz Bauer escreveu:
> > Hi,
> >
> > If pahole -J is used on an ELF that has BTF info from clang, it
> > produces an invalid
> > output. This is because pahole rewrites the .BTF section (which
> > includes a new string
> > table) but it doesn't touch .BTF.ext at all.
>
> > To demonstrate, on a recent check out of bpf-next:
> >     $ cp connect4_prog.o connect4_pahole.o
> >     $ pahole -J connect4_pahole.o
> >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > .BTF.ext=btf-ext.bin connect4_prog.o
> >     $ sha1sum *.bin
> >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> >
> > This problem crops up when compiling old kernels like 4.19 which have
> > an extra pahole
> > build step with clang-10.
>
>
> > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
>
> Agreed.
>
> Longer term pahole needs to generate the .BTF.ext from DWARF, but then,
> if clang is generating it already, why use pahole -J?

Beats me, but then sometimes you don't have control over the workflow, see
my v4.19 kernel example.

>
> Does clang do deduplication for multi-object binaries?
>
> Also its nice to see that the BTF generated ends up with the same
> sha1sum, cool :-)

Unfortunately it's the .BTF.ext section that has the same sha1, because
pahole doesn't touch it ;(

>
> > Best
> > Lorenz
> >
> > --
> > Lorenz Bauer  |  Systems Engineer
> > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> >
> > www.cloudflare.com
>
> --
>
> - Arnaldo



-- 
Lorenz Bauer  |  Systems Engineer
6th Floor, County Hall/The Riverside Building, SE1 7PB, UK

www.cloudflare.com

Re: pahole generates invalid BTF for code compiled with recent clang

[Arnaldo Carvalho de Melo]

Em Wed, Jun 24, 2020 at 05:22:59PM +0100, Lorenz Bauer escreveu:
> On Wed, 24 Jun 2020 at 17:07, Arnaldo Carvalho de Melo <acme-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> >
> > Em Wed, Jun 24, 2020 at 12:05:50PM +0100, Lorenz Bauer escreveu:
> > > Hi,
> > >
> > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > produces an invalid
> > > output. This is because pahole rewrites the .BTF section (which
> > > includes a new string
> > > table) but it doesn't touch .BTF.ext at all.
> >
> > > To demonstrate, on a recent check out of bpf-next:
> > >     $ cp connect4_prog.o connect4_pahole.o
> > >     $ pahole -J connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > .BTF.ext=btf-ext.bin connect4_prog.o
> > >     $ sha1sum *.bin
> > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > >
> > > This problem crops up when compiling old kernels like 4.19 which have
> > > an extra pahole
> > > build step with clang-10.

> > > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> >
> > Agreed.

> > Longer term pahole needs to generate the .BTF.ext from DWARF, but then,
> > if clang is generating it already, why use pahole -J?
 
> Beats me, but then sometimes you don't have control over the workflow, see
> my v4.19 kernel example.
 
> > Does clang do deduplication for multi-object binaries?

> > Also its nice to see that the BTF generated ends up with the same
> > sha1sum, cool :-)
 
> Unfortunately it's the .BTF.ext section that has the same sha1, because
> pahole doesn't touch it ;(

My bad... I guess I saw what I wanted to see... ;-\

- Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Yonghong Song]

On 6/24/20 9:06 AM, Arnaldo Carvalho de Melo wrote:
> Em Wed, Jun 24, 2020 at 12:05:50PM +0100, Lorenz Bauer escreveu:
>> Hi,
>>
>> If pahole -J is used on an ELF that has BTF info from clang, it
>> produces an invalid
>> output. This is because pahole rewrites the .BTF section (which
>> includes a new string
>> table) but it doesn't touch .BTF.ext at all.
>   
>> To demonstrate, on a recent check out of bpf-next:
>>      $ cp connect4_prog.o connect4_pahole.o
>>      $ pahole -J connect4_pahole.o
>>      $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
>> --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
>>      $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
>> .BTF.ext=btf-ext.bin connect4_prog.o
>>      $ sha1sum *.bin
>>      1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
>>      4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
>>      2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
>>      4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
>>
>> This problem crops up when compiling old kernels like 4.19 which have
>> an extra pahole
>> build step with clang-10.
> 
>   
>> I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> 
> Agreed.
> 
> Longer term pahole needs to generate the .BTF.ext from DWARF, but then,
> if clang is generating it already, why use pahole -J?
> 
> Does clang do deduplication for multi-object binaries?

No. It does not. clang did not do static linking for bpf objects 
currently but we may do it in the future...

> 
> Also its nice to see that the BTF generated ends up with the same
> sha1sum, cool :-)
>   
>> Best
>> Lorenz
>>
>> -- 
>> Lorenz Bauer  |  Systems Engineer
>> 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.cloudflare.com&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=VBET_4Cak79EDDrdr9dfzHXXwBqdhb1fmqug2a7lpPc&s=9Wa1gFDt7uR6HK2w9FgzzLXJ2pMq8Ep0i9IXaaqUnb0&e=
> 

Re: pahole generates invalid BTF for code compiled with recent clang

[Andrii Nakryiko]

On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
>
> Hi,
>
> If pahole -J is used on an ELF that has BTF info from clang, it
> produces an invalid
> output. This is because pahole rewrites the .BTF section (which
> includes a new string
> table) but it doesn't touch .BTF.ext at all.

Why do you run `pahole -J` on BPF .o file? Clang already generates
.BTF (and .BTF.ext, of course) for you.

pahole -J is supposed to be used for vmlinux, not for clang-compiled
-target BPF object files.

>
> To demonstrate, on a recent check out of bpf-next:
>     $ cp connect4_prog.o connect4_pahole.o
>     $ pahole -J connect4_pahole.o
>     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
>     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> .BTF.ext=btf-ext.bin connect4_prog.o
>     $ sha1sum *.bin
>     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
>     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
>     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
>     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
>
> This problem crops up when compiling old kernels like 4.19 which have
> an extra pahole
> build step with clang-10.

I was under impression that clang generates .BTF and .BTF.ext only for
-target BPF. In this case, kernel is compiled for "real" target arch,
so there shouldn't be .BTF.ext in the first place? If that's not the
case, then I guess it's a bug in Clang.

>
> I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
>
> Best
> Lorenz
>
> --
> Lorenz Bauer  |  Systems Engineer
> 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
>
> www.cloudflare.com

Re: pahole generates invalid BTF for code compiled with recent clang

[Arnaldo Carvalho de Melo]

Em Wed, Jun 24, 2020 at 10:41:10AM -0700, Andrii Nakryiko escreveu:
> On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> >
> > Hi,
> >
> > If pahole -J is used on an ELF that has BTF info from clang, it
> > produces an invalid
> > output. This is because pahole rewrites the .BTF section (which
> > includes a new string
> > table) but it doesn't touch .BTF.ext at all.
> 
> Why do you run `pahole -J` on BPF .o file? Clang already generates
> .BTF (and .BTF.ext, of course) for you.
> 
> pahole -J is supposed to be used for vmlinux, not for clang-compiled
> -target BPF object files.

yeah, I was thinking this was for a vmlinux generated by clang, which,
from the commands below (the suffix _prog.o) should have told me this is
a target BPF object file.

But then, if one insists for some reason in generating BTF from the
DWARF in a BPF target object file, stripping .BTF.ext, if present, is
the right thing to do at this point.

- Arnaldo
 
> >
> > To demonstrate, on a recent check out of bpf-next:
> >     $ cp connect4_prog.o connect4_pahole.o
> >     $ pahole -J connect4_pahole.o
> >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > .BTF.ext=btf-ext.bin connect4_prog.o
> >     $ sha1sum *.bin
> >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> >
> > This problem crops up when compiling old kernels like 4.19 which have
> > an extra pahole
> > build step with clang-10.
> 
> I was under impression that clang generates .BTF and .BTF.ext only for
> -target BPF. In this case, kernel is compiled for "real" target arch,
> so there shouldn't be .BTF.ext in the first place? If that's not the
> case, then I guess it's a bug in Clang.
> 
> >
> > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> >
> > Best
> > Lorenz
> >
> > --
> > Lorenz Bauer  |  Systems Engineer
> > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> >
> > www.cloudflare.com

-- 

- Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Andrii Nakryiko]

On Wed, Jun 24, 2020 at 10:57 AM Arnaldo Carvalho de Melo
<arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> Em Wed, Jun 24, 2020 at 10:41:10AM -0700, Andrii Nakryiko escreveu:
> > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > >
> > > Hi,
> > >
> > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > produces an invalid
> > > output. This is because pahole rewrites the .BTF section (which
> > > includes a new string
> > > table) but it doesn't touch .BTF.ext at all.
> >
> > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > .BTF (and .BTF.ext, of course) for you.
> >
> > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > -target BPF object files.
>
> yeah, I was thinking this was for a vmlinux generated by clang, which,
> from the commands below (the suffix _prog.o) should have told me this is
> a target BPF object file.
>
> But then, if one insists for some reason in generating BTF from the
> DWARF in a BPF target object file, stripping .BTF.ext, if present, is
> the right thing to do at this point.

I disagree. Those who insist probably have some wrong conceptual
understanding and it's better to fix that (understanding), rather than
lose focus and bend tool to do what it's not supposed to do and
ultimately cause more confusion.

pahole's BTF conversion is really driven towards kernel use-case
(e.g., with global variables, etc). I wouldn't distract ourselves with
supporting de-duplicating BPF object files. Single .o's BTF is already
deduplicated as produced by Clang. Once we add static linking of
multiple BPF .o's (which I hope to start working on very soon), that
de-duplication will be handled automatically by libbpf (and hopefully
integrated into lld as well), among many other things that need to
happen to make static linking work.

>
> - Arnaldo
>
> > >
> > > To demonstrate, on a recent check out of bpf-next:
> > >     $ cp connect4_prog.o connect4_pahole.o
> > >     $ pahole -J connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > .BTF.ext=btf-ext.bin connect4_prog.o
> > >     $ sha1sum *.bin
> > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > >
> > > This problem crops up when compiling old kernels like 4.19 which have
> > > an extra pahole
> > > build step with clang-10.
> >
> > I was under impression that clang generates .BTF and .BTF.ext only for
> > -target BPF. In this case, kernel is compiled for "real" target arch,
> > so there shouldn't be .BTF.ext in the first place? If that's not the
> > case, then I guess it's a bug in Clang.
> >
> > >
> > > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> > >
> > > Best
> > > Lorenz
> > >
> > > --
> > > Lorenz Bauer  |  Systems Engineer
> > > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> > >
> > > www.cloudflare.com
>
> --
>
> - Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Arnaldo Carvalho de Melo]

Em Wed, Jun 24, 2020 at 11:40:21AM -0700, Andrii Nakryiko escreveu:
> On Wed, Jun 24, 2020 at 10:57 AM Arnaldo Carvalho de Melo
> <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > Em Wed, Jun 24, 2020 at 10:41:10AM -0700, Andrii Nakryiko escreveu:
> > > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > > >
> > > > Hi,
> > > >
> > > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > > produces an invalid
> > > > output. This is because pahole rewrites the .BTF section (which
> > > > includes a new string
> > > > table) but it doesn't touch .BTF.ext at all.
> > >
> > > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > > .BTF (and .BTF.ext, of course) for you.
> > >
> > > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > > -target BPF object files.
> >
> > yeah, I was thinking this was for a vmlinux generated by clang, which,
> > from the commands below (the suffix _prog.o) should have told me this is
> > a target BPF object file.
> >
> > But then, if one insists for some reason in generating BTF from the
> > DWARF in a BPF target object file, stripping .BTF.ext, if present, is
> > the right thing to do at this point.
> 
> I disagree. Those who insist probably have some wrong conceptual
> understanding and it's better to fix that (understanding), rather than
> lose focus and bend tool to do what it's not supposed to do and
> ultimately cause more confusion.

So we can instead notice the presence of .BTF.ext when the user calls
'pahole -J' on a target BPF object file and bail out, only allowing it
to convert from DWARF to BTF and thus encode the .BTF elf section when
.BTF.ext isn't present, as we can't easily figure out if the present of
just .BTF section was done by clang or pahole on a BTF target object
file built without -g.

- Arnaldo
 
> pahole's BTF conversion is really driven towards kernel use-case
> (e.g., with global variables, etc). I wouldn't distract ourselves with
> supporting de-duplicating BPF object files. Single .o's BTF is already
> deduplicated as produced by Clang. Once we add static linking of
> multiple BPF .o's (which I hope to start working on very soon), that
> de-duplication will be handled automatically by libbpf (and hopefully
> integrated into lld as well), among many other things that need to
> happen to make static linking work.
> 
> >
> > - Arnaldo
> >
> > > >
> > > > To demonstrate, on a recent check out of bpf-next:
> > > >     $ cp connect4_prog.o connect4_pahole.o
> > > >     $ pahole -J connect4_pahole.o
> > > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > > .BTF.ext=btf-ext.bin connect4_prog.o
> > > >     $ sha1sum *.bin
> > > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > > >
> > > > This problem crops up when compiling old kernels like 4.19 which have
> > > > an extra pahole
> > > > build step with clang-10.
> > >
> > > I was under impression that clang generates .BTF and .BTF.ext only for
> > > -target BPF. In this case, kernel is compiled for "real" target arch,
> > > so there shouldn't be .BTF.ext in the first place? If that's not the
> > > case, then I guess it's a bug in Clang.
> > >
> > > >
> > > > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> > > >
> > > > Best
> > > > Lorenz
> > > >
> > > > --
> > > > Lorenz Bauer  |  Systems Engineer
> > > > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> > > >
> > > > www.cloudflare.com
> >
> > --
> >
> > - Arnaldo

-- 

- Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Andrii Nakryiko]

On Wed, Jun 24, 2020 at 11:57 AM Arnaldo Carvalho de Melo
<arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> Em Wed, Jun 24, 2020 at 11:40:21AM -0700, Andrii Nakryiko escreveu:
> > On Wed, Jun 24, 2020 at 10:57 AM Arnaldo Carvalho de Melo
> > <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > >
> > > Em Wed, Jun 24, 2020 at 10:41:10AM -0700, Andrii Nakryiko escreveu:
> > > > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > > > produces an invalid
> > > > > output. This is because pahole rewrites the .BTF section (which
> > > > > includes a new string
> > > > > table) but it doesn't touch .BTF.ext at all.
> > > >
> > > > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > > > .BTF (and .BTF.ext, of course) for you.
> > > >
> > > > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > > > -target BPF object files.
> > >
> > > yeah, I was thinking this was for a vmlinux generated by clang, which,
> > > from the commands below (the suffix _prog.o) should have told me this is
> > > a target BPF object file.
> > >
> > > But then, if one insists for some reason in generating BTF from the
> > > DWARF in a BPF target object file, stripping .BTF.ext, if present, is
> > > the right thing to do at this point.
> >
> > I disagree. Those who insist probably have some wrong conceptual
> > understanding and it's better to fix that (understanding), rather than
> > lose focus and bend tool to do what it's not supposed to do and
> > ultimately cause more confusion.
>
> So we can instead notice the presence of .BTF.ext when the user calls
> 'pahole -J' on a target BPF object file and bail out, only allowing it
> to convert from DWARF to BTF and thus encode the .BTF elf section when
> .BTF.ext isn't present, as we can't easily figure out if the present of
> just .BTF section was done by clang or pahole on a BTF target object
> file built without -g.

Can't we check ELF's target machine and reject if it's a BPF one?
Someday we might also support "cross-compilation" to be able to dedup
arm ELF from x86 machine. It's sort of ok today for little-endian
ARMs, so maybe not outright reject if architecture is not the same as
the local one?

>
> - Arnaldo
>
> > pahole's BTF conversion is really driven towards kernel use-case
> > (e.g., with global variables, etc). I wouldn't distract ourselves with
> > supporting de-duplicating BPF object files. Single .o's BTF is already
> > deduplicated as produced by Clang. Once we add static linking of
> > multiple BPF .o's (which I hope to start working on very soon), that
> > de-duplication will be handled automatically by libbpf (and hopefully
> > integrated into lld as well), among many other things that need to
> > happen to make static linking work.
> >
> > >
> > > - Arnaldo
> > >
> > > > >
> > > > > To demonstrate, on a recent check out of bpf-next:
> > > > >     $ cp connect4_prog.o connect4_pahole.o
> > > > >     $ pahole -J connect4_pahole.o
> > > > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > > > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > > > .BTF.ext=btf-ext.bin connect4_prog.o
> > > > >     $ sha1sum *.bin
> > > > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > > > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > > > >
> > > > > This problem crops up when compiling old kernels like 4.19 which have
> > > > > an extra pahole
> > > > > build step with clang-10.
> > > >
> > > > I was under impression that clang generates .BTF and .BTF.ext only for
> > > > -target BPF. In this case, kernel is compiled for "real" target arch,
> > > > so there shouldn't be .BTF.ext in the first place? If that's not the
> > > > case, then I guess it's a bug in Clang.
> > > >
> > > > >
> > > > > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> > > > >
> > > > > Best
> > > > > Lorenz
> > > > >
> > > > > --
> > > > > Lorenz Bauer  |  Systems Engineer
> > > > > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> > > > >
> > > > > www.cloudflare.com
> > >
> > > --
> > >
> > > - Arnaldo
>
> --
>
> - Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Arnaldo Carvalho de Melo]

Em Wed, Jun 24, 2020 at 12:06:24PM -0700, Andrii Nakryiko escreveu:
> On Wed, Jun 24, 2020 at 11:57 AM Arnaldo Carvalho de Melo
> <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > Em Wed, Jun 24, 2020 at 11:40:21AM -0700, Andrii Nakryiko escreveu:
> > > On Wed, Jun 24, 2020 at 10:57 AM Arnaldo Carvalho de Melo
> > > <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > > >
> > > > Em Wed, Jun 24, 2020 at 10:41:10AM -0700, Andrii Nakryiko escreveu:
> > > > > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > > > > produces an invalid
> > > > > > output. This is because pahole rewrites the .BTF section (which
> > > > > > includes a new string
> > > > > > table) but it doesn't touch .BTF.ext at all.
> > > > >
> > > > > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > > > > .BTF (and .BTF.ext, of course) for you.
> > > > >
> > > > > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > > > > -target BPF object files.
> > > >
> > > > yeah, I was thinking this was for a vmlinux generated by clang, which,
> > > > from the commands below (the suffix _prog.o) should have told me this is
> > > > a target BPF object file.
> > > >
> > > > But then, if one insists for some reason in generating BTF from the
> > > > DWARF in a BPF target object file, stripping .BTF.ext, if present, is
> > > > the right thing to do at this point.
> > >
> > > I disagree. Those who insist probably have some wrong conceptual
> > > understanding and it's better to fix that (understanding), rather than
> > > lose focus and bend tool to do what it's not supposed to do and
> > > ultimately cause more confusion.
> >
> > So we can instead notice the presence of .BTF.ext when the user calls
> > 'pahole -J' on a target BPF object file and bail out, only allowing it
> > to convert from DWARF to BTF and thus encode the .BTF elf section when
> > .BTF.ext isn't present, as we can't easily figure out if the present of
> > just .BTF section was done by clang or pahole on a BTF target object
> > file built without -g.
> 
> Can't we check ELF's target machine and reject if it's a BPF one?

I think there is value in allowing pahole to convert DWARF to BTF even
for a BPF target object file, say in some case people may think clang is
not generating correct BTF so one may want to see what pahole generates
and compare.

> Someday we might also support "cross-compilation" to be able to dedup
> arm ELF from x86 machine. It's sort of ok today for little-endian
> ARMs, so maybe not outright reject if architecture is not the same as
> the local one?

I think outright reject if arch is not t he same it not necessary.

We may warn the user that using -g in clang is the preferred method for
generating BTF, wdyt?

- Arnaldo
 
> >
> > - Arnaldo
> >
> > > pahole's BTF conversion is really driven towards kernel use-case
> > > (e.g., with global variables, etc). I wouldn't distract ourselves with
> > > supporting de-duplicating BPF object files. Single .o's BTF is already
> > > deduplicated as produced by Clang. Once we add static linking of
> > > multiple BPF .o's (which I hope to start working on very soon), that
> > > de-duplication will be handled automatically by libbpf (and hopefully
> > > integrated into lld as well), among many other things that need to
> > > happen to make static linking work.
> > >
> > > >
> > > > - Arnaldo
> > > >
> > > > > >
> > > > > > To demonstrate, on a recent check out of bpf-next:
> > > > > >     $ cp connect4_prog.o connect4_pahole.o
> > > > > >     $ pahole -J connect4_pahole.o
> > > > > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > > > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > > > > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > > > > .BTF.ext=btf-ext.bin connect4_prog.o
> > > > > >     $ sha1sum *.bin
> > > > > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > > > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > > > > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > > > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > > > > >
> > > > > > This problem crops up when compiling old kernels like 4.19 which have
> > > > > > an extra pahole
> > > > > > build step with clang-10.
> > > > >
> > > > > I was under impression that clang generates .BTF and .BTF.ext only for
> > > > > -target BPF. In this case, kernel is compiled for "real" target arch,
> > > > > so there shouldn't be .BTF.ext in the first place? If that's not the
> > > > > case, then I guess it's a bug in Clang.
> > > > >
> > > > > >
> > > > > > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> > > > > >
> > > > > > Best
> > > > > > Lorenz
> > > > > >
> > > > > > --
> > > > > > Lorenz Bauer  |  Systems Engineer
> > > > > > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> > > > > >
> > > > > > www.cloudflare.com
> > > >
> > > > --
> > > >
> > > > - Arnaldo
> >
> > --
> >
> > - Arnaldo

-- 

- Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Andrii Nakryiko]

On Wed, Jun 24, 2020 at 12:12 PM Arnaldo Carvalho de Melo
<arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> Em Wed, Jun 24, 2020 at 12:06:24PM -0700, Andrii Nakryiko escreveu:
> > On Wed, Jun 24, 2020 at 11:57 AM Arnaldo Carvalho de Melo
> > <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > >
> > > Em Wed, Jun 24, 2020 at 11:40:21AM -0700, Andrii Nakryiko escreveu:
> > > > On Wed, Jun 24, 2020 at 10:57 AM Arnaldo Carvalho de Melo
> > > > <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > > > >
> > > > > Em Wed, Jun 24, 2020 at 10:41:10AM -0700, Andrii Nakryiko escreveu:
> > > > > > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > > > > > produces an invalid
> > > > > > > output. This is because pahole rewrites the .BTF section (which
> > > > > > > includes a new string
> > > > > > > table) but it doesn't touch .BTF.ext at all.
> > > > > >
> > > > > > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > > > > > .BTF (and .BTF.ext, of course) for you.
> > > > > >
> > > > > > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > > > > > -target BPF object files.
> > > > >
> > > > > yeah, I was thinking this was for a vmlinux generated by clang, which,
> > > > > from the commands below (the suffix _prog.o) should have told me this is
> > > > > a target BPF object file.
> > > > >
> > > > > But then, if one insists for some reason in generating BTF from the
> > > > > DWARF in a BPF target object file, stripping .BTF.ext, if present, is
> > > > > the right thing to do at this point.
> > > >
> > > > I disagree. Those who insist probably have some wrong conceptual
> > > > understanding and it's better to fix that (understanding), rather than
> > > > lose focus and bend tool to do what it's not supposed to do and
> > > > ultimately cause more confusion.
> > >
> > > So we can instead notice the presence of .BTF.ext when the user calls
> > > 'pahole -J' on a target BPF object file and bail out, only allowing it
> > > to convert from DWARF to BTF and thus encode the .BTF elf section when
> > > .BTF.ext isn't present, as we can't easily figure out if the present of
> > > just .BTF section was done by clang or pahole on a BTF target object
> > > file built without -g.
> >
> > Can't we check ELF's target machine and reject if it's a BPF one?
>
> I think there is value in allowing pahole to convert DWARF to BTF even
> for a BPF target object file, say in some case people may think clang is
> not generating correct BTF so one may want to see what pahole generates
> and compare.

sure, and will the warning for wrong architecture would give a hint
that it's not the right thing to do, probably. Or we could have more
specific message for BPF target. I don't care all that much.


>
> > Someday we might also support "cross-compilation" to be able to dedup
> > arm ELF from x86 machine. It's sort of ok today for little-endian
> > ARMs, so maybe not outright reject if architecture is not the same as
> > the local one?
>
> I think outright reject if arch is not t he same it not necessary.
>
> We may warn the user that using -g in clang is the preferred method for
> generating BTF, wdyt?
>

sounds reasonable

> - Arnaldo
>
> > >
> > > - Arnaldo
> > >
> > > > pahole's BTF conversion is really driven towards kernel use-case
> > > > (e.g., with global variables, etc). I wouldn't distract ourselves with
> > > > supporting de-duplicating BPF object files. Single .o's BTF is already
> > > > deduplicated as produced by Clang. Once we add static linking of
> > > > multiple BPF .o's (which I hope to start working on very soon), that
> > > > de-duplication will be handled automatically by libbpf (and hopefully
> > > > integrated into lld as well), among many other things that need to
> > > > happen to make static linking work.
> > > >
> > > > >
> > > > > - Arnaldo
> > > > >
> > > > > > >
> > > > > > > To demonstrate, on a recent check out of bpf-next:
> > > > > > >     $ cp connect4_prog.o connect4_pahole.o
> > > > > > >     $ pahole -J connect4_pahole.o
> > > > > > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > > > > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > > > > > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > > > > > .BTF.ext=btf-ext.bin connect4_prog.o
> > > > > > >     $ sha1sum *.bin
> > > > > > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > > > > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > > > > > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > > > > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > > > > > >
> > > > > > > This problem crops up when compiling old kernels like 4.19 which have
> > > > > > > an extra pahole
> > > > > > > build step with clang-10.
> > > > > >
> > > > > > I was under impression that clang generates .BTF and .BTF.ext only for
> > > > > > -target BPF. In this case, kernel is compiled for "real" target arch,
> > > > > > so there shouldn't be .BTF.ext in the first place? If that's not the
> > > > > > case, then I guess it's a bug in Clang.
> > > > > >
> > > > > > >
> > > > > > > I think a possible fix is to strip .BTF.ext if .BTF is rewritten.
> > > > > > >
> > > > > > > Best
> > > > > > > Lorenz
> > > > > > >
> > > > > > > --
> > > > > > > Lorenz Bauer  |  Systems Engineer
> > > > > > > 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
> > > > > > >
> > > > > > > www.cloudflare.com
> > > > >
> > > > > --
> > > > >
> > > > > - Arnaldo
> > >
> > > --
> > >
> > > - Arnaldo
>
> --
>
> - Arnaldo

Re: pahole generates invalid BTF for code compiled with recent clang

[Lorenz Bauer]

On Wed, 24 Jun 2020 at 18:41, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> >
> > Hi,
> >
> > If pahole -J is used on an ELF that has BTF info from clang, it
> > produces an invalid
> > output. This is because pahole rewrites the .BTF section (which
> > includes a new string
> > table) but it doesn't touch .BTF.ext at all.
>
> Why do you run `pahole -J` on BPF .o file? Clang already generates
> .BTF (and .BTF.ext, of course) for you.

You're missing the point. The kernel build system does it. Try the following:

* Get the v4.19 sources
* Make sure that clang --version is 10
* Make sure you have pahole (I used v1.17)
* Build selftests

The resulting object files will have bogus .BTF.ext sections due the
bug I have described. Does it make sense to run pahole -J on these?
No, but it still happens.

I think it's reasonable to expect to get valid BPF ELFs out of this process.

>
> pahole -J is supposed to be used for vmlinux, not for clang-compiled
> -target BPF object files.
>
> >
> > To demonstrate, on a recent check out of bpf-next:
> >     $ cp connect4_prog.o connect4_pahole.o
> >     $ pahole -J connect4_pahole.o
> >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > .BTF.ext=btf-ext.bin connect4_prog.o
> >     $ sha1sum *.bin
> >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> >
> > This problem crops up when compiling old kernels like 4.19 which have
> > an extra pahole
> > build step with clang-10.
>
> I was under impression that clang generates .BTF and .BTF.ext only for
> -target BPF. In this case, kernel is compiled for "real" target arch,
> so there shouldn't be .BTF.ext in the first place? If that's not the
> case, then I guess it's a bug in Clang.

connect4_prog.o is BPF:

$ readelf -h connect4_prog.o | grep BPF
  Machine:                           Linux BPF

Maybe I misunderstand what you're trying to say.

Best
Lorenz

-- 
Lorenz Bauer  |  Systems Engineer
6th Floor, County Hall/The Riverside Building, SE1 7PB, UK

www.cloudflare.com

Re: pahole generates invalid BTF for code compiled with recent clang

[Andrii Nakryiko]

On Thu, Jun 25, 2020 at 2:25 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
>
> On Wed, 24 Jun 2020 at 18:41, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > >
> > > Hi,
> > >
> > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > produces an invalid
> > > output. This is because pahole rewrites the .BTF section (which
> > > includes a new string
> > > table) but it doesn't touch .BTF.ext at all.
> >
> > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > .BTF (and .BTF.ext, of course) for you.
>
> You're missing the point. The kernel build system does it. Try the following:

Yeah, I clearly am, because "compiling old kernels like 4.19" made me
think that we are talking about building kernel, not selftests.

>
> * Get the v4.19 sources
> * Make sure that clang --version is 10
> * Make sure you have pahole (I used v1.17)
> * Build selftests
>
> The resulting object files will have bogus .BTF.ext sections due the
> bug I have described. Does it make sense to run pahole -J on these?
> No, but it still happens.
>

Yeah, because back in the day Clang didn't know how to generate .BTF,
so using pahole to generate .BTF for BPF object files was a solution.

> I think it's reasonable to expect to get valid BPF ELFs out of this process.

We should probably update Makefile for old kernel selftest to not call
pahole -J, if Clang is recent enough and/or if .o file already has
.BTF. That shouldn't be hard.

>
> >
> > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > -target BPF object files.
> >
> > >
> > > To demonstrate, on a recent check out of bpf-next:
> > >     $ cp connect4_prog.o connect4_pahole.o
> > >     $ pahole -J connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > .BTF.ext=btf-ext.bin connect4_prog.o
> > >     $ sha1sum *.bin
> > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > >
> > > This problem crops up when compiling old kernels like 4.19 which have
> > > an extra pahole
> > > build step with clang-10.
> >
> > I was under impression that clang generates .BTF and .BTF.ext only for
> > -target BPF. In this case, kernel is compiled for "real" target arch,
> > so there shouldn't be .BTF.ext in the first place? If that's not the
> > case, then I guess it's a bug in Clang.
>
> connect4_prog.o is BPF:
>
> $ readelf -h connect4_prog.o | grep BPF
>   Machine:                           Linux BPF
>
> Maybe I misunderstand what you're trying to say.
>

I was talking/thinking about building kernel, you were talking about
building BPF object files in selftests. Just to avoid confusion in the
future, let's not talk about compiling kernel, when we are talking
about compiling selftests. See my suggestion for a fix above. Would
that work?

> Best
> Lorenz
>
> --
> Lorenz Bauer  |  Systems Engineer
> 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK
>
> www.cloudflare.com

Re: pahole generates invalid BTF for code compiled with recent clang

[Lorenz Bauer]

On Thu, 25 Jun 2020 at 17:56, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> On Thu, Jun 25, 2020 at 2:25 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> >
> > On Wed, 24 Jun 2020 at 18:41, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > >
> > > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > > >
> > > > Hi,
> > > >
> > > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > > produces an invalid
> > > > output. This is because pahole rewrites the .BTF section (which
> > > > includes a new string
> > > > table) but it doesn't touch .BTF.ext at all.
> > >
> > > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > > .BTF (and .BTF.ext, of course) for you.
> >
> > You're missing the point. The kernel build system does it. Try the following:
>
> Yeah, I clearly am, because "compiling old kernels like 4.19" made me
> think that we are talking about building kernel, not selftests.
>
> >
> > * Get the v4.19 sources
> > * Make sure that clang --version is 10
> > * Make sure you have pahole (I used v1.17)
> > * Build selftests
> >
> > The resulting object files will have bogus .BTF.ext sections due the
> > bug I have described. Does it make sense to run pahole -J on these?
> > No, but it still happens.
> >
>
> Yeah, because back in the day Clang didn't know how to generate .BTF,
> so using pahole to generate .BTF for BPF object files was a solution.
>
> > I think it's reasonable to expect to get valid BPF ELFs out of this process.
>
> We should probably update Makefile for old kernel selftest to not call
> pahole -J, if Clang is recent enough and/or if .o file already has
> .BTF. That shouldn't be hard.

Yeah, I think that would work!

>
> >
> > >
> > > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > > -target BPF object files.
> > >
> > > >
> > > > To demonstrate, on a recent check out of bpf-next:
> > > >     $ cp connect4_prog.o connect4_pahole.o
> > > >     $ pahole -J connect4_pahole.o
> > > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > > .BTF.ext=btf-ext.bin connect4_prog.o
> > > >     $ sha1sum *.bin
> > > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > > >
> > > > This problem crops up when compiling old kernels like 4.19 which have
> > > > an extra pahole
> > > > build step with clang-10.
> > >
> > > I was under impression that clang generates .BTF and .BTF.ext only for
> > > -target BPF. In this case, kernel is compiled for "real" target arch,
> > > so there shouldn't be .BTF.ext in the first place? If that's not the
> > > case, then I guess it's a bug in Clang.
> >
> > connect4_prog.o is BPF:
> >
> > $ readelf -h connect4_prog.o | grep BPF
> >   Machine:                           Linux BPF
> >
> > Maybe I misunderstand what you're trying to say.
> >
>
> I was talking/thinking about building kernel, you were talking about
> building BPF object files in selftests. Just to avoid confusion in the
> future, let's not talk about compiling kernel, when we are talking
> about compiling selftests.

Point taken :)

Lorenz

-- 
Lorenz Bauer  |  Systems Engineer
6th Floor, County Hall/The Riverside Building, SE1 7PB, UK

www.cloudflare.com