Dwarves Archive on lore.kernel.org
 help / color / Atom feed
From: Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org>
To: Andrii Nakryiko
	<andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: bpf <bpf-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	kernel-team <kernel-team-lDpJ742SOEtZroRs9YW3xA@public.gmane.org>
Subject: Re: pahole generates invalid BTF for code compiled with recent clang
Date: Tue, 30 Jun 2020 16:52:16 +0100
Message-ID: <CACAyw9-pPbTVwQ0AM=1-7SQmFdU6EF127vcqy5pmPqL7gQP3nQ@mail.gmail.com> (raw)
In-Reply-To: <CAEf4BzYjge6fijFadwuuHf-vr2VUqneT5b0k-GgQSkLMTj=UAA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On Thu, 25 Jun 2020 at 17:56, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> On Thu, Jun 25, 2020 at 2:25 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> >
> > On Wed, 24 Jun 2020 at 18:41, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > >
> > > On Wed, Jun 24, 2020 at 4:07 AM Lorenz Bauer <lmb-lDpJ742SOEtZroRs9YW3xA@public.gmane.org> wrote:
> > > >
> > > > Hi,
> > > >
> > > > If pahole -J is used on an ELF that has BTF info from clang, it
> > > > produces an invalid
> > > > output. This is because pahole rewrites the .BTF section (which
> > > > includes a new string
> > > > table) but it doesn't touch .BTF.ext at all.
> > >
> > > Why do you run `pahole -J` on BPF .o file? Clang already generates
> > > .BTF (and .BTF.ext, of course) for you.
> >
> > You're missing the point. The kernel build system does it. Try the following:
>
> Yeah, I clearly am, because "compiling old kernels like 4.19" made me
> think that we are talking about building kernel, not selftests.
>
> >
> > * Get the v4.19 sources
> > * Make sure that clang --version is 10
> > * Make sure you have pahole (I used v1.17)
> > * Build selftests
> >
> > The resulting object files will have bogus .BTF.ext sections due the
> > bug I have described. Does it make sense to run pahole -J on these?
> > No, but it still happens.
> >
>
> Yeah, because back in the day Clang didn't know how to generate .BTF,
> so using pahole to generate .BTF for BPF object files was a solution.
>
> > I think it's reasonable to expect to get valid BPF ELFs out of this process.
>
> We should probably update Makefile for old kernel selftest to not call
> pahole -J, if Clang is recent enough and/or if .o file already has
> .BTF. That shouldn't be hard.

Yeah, I think that would work!

>
> >
> > >
> > > pahole -J is supposed to be used for vmlinux, not for clang-compiled
> > > -target BPF object files.
> > >
> > > >
> > > > To demonstrate, on a recent check out of bpf-next:
> > > >     $ cp connect4_prog.o connect4_pahole.o
> > > >     $ pahole -J connect4_pahole.o
> > > >     $ llvm-objcopy-10 --dump-section .BTF=pahole-btf.bin
> > > > --dump-section .BTF.ext=pahole-btf-ext.bin connect4_pahole.o
> > > >     $ llvm-objcopy-10 --dump-section .BTF=btf.bin --dump-section
> > > > .BTF.ext=btf-ext.bin connect4_prog.o
> > > >     $ sha1sum *.bin
> > > >     1b5c7407dd9fd13f969931d32f6b864849e66a68  btf.bin
> > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  btf-ext.bin
> > > >     2a60767a3a037de66a8d963110601769fa0f198e  pahole-btf.bin
> > > >     4c43efcc86d3cd908ddc77c15fc4a35af38d842b  pahole-btf-ext.bin
> > > >
> > > > This problem crops up when compiling old kernels like 4.19 which have
> > > > an extra pahole
> > > > build step with clang-10.
> > >
> > > I was under impression that clang generates .BTF and .BTF.ext only for
> > > -target BPF. In this case, kernel is compiled for "real" target arch,
> > > so there shouldn't be .BTF.ext in the first place? If that's not the
> > > case, then I guess it's a bug in Clang.
> >
> > connect4_prog.o is BPF:
> >
> > $ readelf -h connect4_prog.o | grep BPF
> >   Machine:                           Linux BPF
> >
> > Maybe I misunderstand what you're trying to say.
> >
>
> I was talking/thinking about building kernel, you were talking about
> building BPF object files in selftests. Just to avoid confusion in the
> future, let's not talk about compiling kernel, when we are talking
> about compiling selftests.

Point taken :)

Lorenz

-- 
Lorenz Bauer  |  Systems Engineer
6th Floor, County Hall/The Riverside Building, SE1 7PB, UK

www.cloudflare.com

      parent reply index

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-24 11:05 Lorenz Bauer
     [not found] ` <CACAyw9-cinpz=U+8tjV-GMWuth71jrOYLQ05Q7_c34TCeMJxMg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-24 16:06   ` Arnaldo Carvalho de Melo
     [not found]     ` <20200624160659.GA20203-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-06-24 16:22       ` Lorenz Bauer
     [not found]         ` <CACAyw9-zLLDJ4vXo7jGS_XoYsiiv4c5NmUCjCnAf0eZBXU3dVA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-24 17:23           ` Arnaldo Carvalho de Melo
2020-06-24 17:05       ` Yonghong Song
2020-06-24 17:41   ` Andrii Nakryiko
     [not found]     ` <CAEf4BzbSc-wykq1_62CQwtszO+76rkudz_B=GkzE6ZheMUAusw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-24 17:57       ` Arnaldo Carvalho de Melo
     [not found]         ` <20200624175754.GD20203-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-06-24 18:40           ` Andrii Nakryiko
     [not found]             ` <CAEf4BzY8b71tE5B4rw5sfy=xajtgqUGHVaoHNf_YzVtQ9aLCBg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-24 18:57               ` Arnaldo Carvalho de Melo
     [not found]                 ` <20200624185737.GA25807-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-06-24 19:06                   ` Andrii Nakryiko
     [not found]                     ` <CAEf4Bza=ZT1yZvoJNMK72pYm6VGwGp22detc7kgC_24OBt4-FA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-24 19:11                       ` Arnaldo Carvalho de Melo
     [not found]                         ` <20200624191159.GB25807-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-06-24 19:25                           ` Andrii Nakryiko
2020-06-25  9:25       ` Lorenz Bauer
     [not found]         ` <CACAyw98ojwGjQm+Xk+_-B8Ah-hEt-Tgv_LQ1BdH4yBLYgVwpiA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-25 16:56           ` Andrii Nakryiko
     [not found]             ` <CAEf4BzYjge6fijFadwuuHf-vr2VUqneT5b0k-GgQSkLMTj=UAA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30 15:52               ` Lorenz Bauer [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CACAyw9-pPbTVwQ0AM=1-7SQmFdU6EF127vcqy5pmPqL7gQP3nQ@mail.gmail.com' \
    --to=lmb-ldpj742soetzrors9yw3xa@public.gmane.org \
    --cc=andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=bpf-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=kernel-team-lDpJ742SOEtZroRs9YW3xA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Dwarves Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dwarves/0 dwarves/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dwarves dwarves/ https://lore.kernel.org/dwarves \
		dwarves@vger.kernel.org
	public-inbox-index dwarves

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.dwarves


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git