BPF Archive on lore.kernel.org
 help / color / Atom feed
* Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
@ 2021-01-11 12:48 Sedat Dilek
  2021-01-11 16:05 ` Yonghong Song
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-11 12:48 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Masahiro Yamada
  Cc: bpf, linux-kbuild

Hi BPF maintainers and Mashiro,

Debian started to use CONFIG_DEBUG_INFO_BTF=y.

My kernel-build fails like this:

+ info BTFIDS vmlinux
+ [  != silent_ ]
+ printf   %-7s %s\n BTFIDS vmlinux
 BTFIDS  vmlinux
+ ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
FAILED: load BTF from vmlinux: Invalid argument

The root cause is my selfmade LLVM toolchain has no BPF support.

$ which llc
/home/dileks/src/llvm-toolchain/install/bin/llc

$ llc --version
LLVM (http://llvm.org/):
 LLVM version 11.0.1
 Optimized build.
 Default target: x86_64-unknown-linux-gnu
 Host CPU: sandybridge

 Registered Targets:
   x86    - 32-bit X86: Pentium-Pro and above
   x86-64 - 64-bit X86: EM64T and AMD64

Debian's llc-11 shows me BPF support is built-in.

I see the breakag approx. 3 hours after the start of my kernel-build -
in the stage "vmlinux".
After 2 faulures in my build (2x 3 hours of build-time) I have still
no finished Linux v5.11-rc3 kernel.
This is a bit frustrating.

What about doing pre-checks - means before doing a single line of
compilation - to check for:
1. Required binaries
2. Required support of whatever feature in compiler, linker, toolchain etc.

Recently, I fell over depmod binary not found in my PATH - in one of
the last steps (modfinal) of the kernel build.

Any ideas to improve the situation?
( ...and please no RTFM, see links below. )

Thanks.

Regards,
- Sedat -


[0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
[1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
[2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
[3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 12:48 Check pahole availibity and BPF support of toolchain before starting a Linux kernel build Sedat Dilek
@ 2021-01-11 16:05 ` Yonghong Song
  2021-01-11 17:56   ` Sedat Dilek
  0 siblings, 1 reply; 19+ messages in thread
From: Yonghong Song @ 2021-01-11 16:05 UTC (permalink / raw)
  To: sedat.dilek, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend,
	KP Singh, Masahiro Yamada
  Cc: bpf, linux-kbuild



On 1/11/21 4:48 AM, Sedat Dilek wrote:
> Hi BPF maintainers and Mashiro,
> 
> Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> 
> My kernel-build fails like this:
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>   BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument
> 
> The root cause is my selfmade LLVM toolchain has no BPF support.

linux build should depend on LLVM toolchain unless you use LLVM to build 
kernel.

> 
> $ which llc
> /home/dileks/src/llvm-toolchain/install/bin/llc
> 
> $ llc --version
> LLVM (http://llvm.org/ ):
>   LLVM version 11.0.1
>   Optimized build.
>   Default target: x86_64-unknown-linux-gnu
>   Host CPU: sandybridge
> 
>   Registered Targets:
>     x86    - 32-bit X86: Pentium-Pro and above
>     x86-64 - 64-bit X86: EM64T and AMD64
> 
> Debian's llc-11 shows me BPF support is built-in.
> 
> I see the breakag approx. 3 hours after the start of my kernel-build -
> in the stage "vmlinux".
> After 2 faulures in my build (2x 3 hours of build-time) I have still
> no finished Linux v5.11-rc3 kernel.
> This is a bit frustrating.

You mean "BTFIDS  vmlinux" takes more than 3 hours here?
Maybe a bug in resolve_btfids due to somehow different ELF format
resolve_btfids need to handle?

> 
> What about doing pre-checks - means before doing a single line of
> compilation - to check for:
> 1. Required binaries
> 2. Required support of whatever feature in compiler, linker, toolchain etc.
> 
> Recently, I fell over depmod binary not found in my PATH - in one of
> the last steps (modfinal) of the kernel build.
> 
> Any ideas to improve the situation?
> ( ...and please no RTFM, see links below. )
> 
> Thanks.
> 
> Regards,
> - Sedat -
> 
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> 

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 16:05 ` Yonghong Song
@ 2021-01-11 17:56   ` Sedat Dilek
  2021-01-11 21:03     ` Andrii Nakryiko
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-11 17:56 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, linux-kbuild, Tom Stellard

On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 1/11/21 4:48 AM, Sedat Dilek wrote:
> > Hi BPF maintainers and Mashiro,
> >
> > Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> >
> > My kernel-build fails like this:
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >   BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
> >
> > The root cause is my selfmade LLVM toolchain has no BPF support.
>
> linux build should depend on LLVM toolchain unless you use LLVM to build
> kernel.
>
> >
> > $ which llc
> > /home/dileks/src/llvm-toolchain/install/bin/llc
> >
> > $ llc --version
> > LLVM (http://llvm.org/ ):
> >   LLVM version 11.0.1
> >   Optimized build.
> >   Default target: x86_64-unknown-linux-gnu
> >   Host CPU: sandybridge
> >
> >   Registered Targets:
> >     x86    - 32-bit X86: Pentium-Pro and above
> >     x86-64 - 64-bit X86: EM64T and AMD64
> >
> > Debian's llc-11 shows me BPF support is built-in.
> >
> > I see the breakag approx. 3 hours after the start of my kernel-build -
> > in the stage "vmlinux".
> > After 2 faulures in my build (2x 3 hours of build-time) I have still
> > no finished Linux v5.11-rc3 kernel.
> > This is a bit frustrating.
>
> You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> Maybe a bug in resolve_btfids due to somehow different ELF format
> resolve_btfids need to handle?
>

[ CC Tom ]

OMG no.

3 hours up to running scripts/link-vmlinux.sh.

In the meantime I have built a LLVM toolchain with BPF support.

$ llc --version
LLVM (http://llvm.org/):
 LLVM version 11.0.1
 Optimized build.
 Default target: x86_64-unknown-linux-gnu
 Host CPU: sandybridge

 Registered Targets:
   bpf    - BPF (host endian)
   bpfeb  - BPF (big endian)
   bpfel  - BPF (little endian)
   x86    - 32-bit X86: Pentium-Pro and above
   x86-64 - 64-bit X86: EM64T and AMD64

Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
"I ran into this same bug trying to build the Fedora kernel. The
problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
looks to me like a bug in pahole."

pahole ToT (post v1.19) offers some BTF fixes - I have manually build
and use it.

Building a new Linux-kernel...

- Sedat -

[1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
[2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
[3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553



> >
> > What about doing pre-checks - means before doing a single line of
> > compilation - to check for:
> > 1. Required binaries
> > 2. Required support of whatever feature in compiler, linker, toolchain etc.
> >
> > Recently, I fell over depmod binary not found in my PATH - in one of
> > the last steps (modfinal) of the kernel build.
> >
> > Any ideas to improve the situation?
> > ( ...and please no RTFM, see links below. )
> >
> > Thanks.
> >
> > Regards,
> > - Sedat -
> >
> >
> > [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> > [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> > [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> > [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> >

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 17:56   ` Sedat Dilek
@ 2021-01-11 21:03     ` Andrii Nakryiko
       [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
  0 siblings, 1 reply; 19+ messages in thread
From: Andrii Nakryiko @ 2021-01-11 21:03 UTC (permalink / raw)
  To: sedat.dilek, Jiri Olsa
  Cc: Yonghong Song, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend,
	KP Singh, Masahiro Yamada, bpf, Linux Kbuild mailing list,
	Tom Stellard

On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
> >
> >
> >
> > On 1/11/21 4:48 AM, Sedat Dilek wrote:
> > > Hi BPF maintainers and Mashiro,
> > >
> > > Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> > >
> > > My kernel-build fails like this:
> > >
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >   BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > The root cause is my selfmade LLVM toolchain has no BPF support.
> >
> > linux build should depend on LLVM toolchain unless you use LLVM to build
> > kernel.
> >
> > >
> > > $ which llc
> > > /home/dileks/src/llvm-toolchain/install/bin/llc
> > >
> > > $ llc --version
> > > LLVM (http://llvm.org/ ):
> > >   LLVM version 11.0.1
> > >   Optimized build.
> > >   Default target: x86_64-unknown-linux-gnu
> > >   Host CPU: sandybridge
> > >
> > >   Registered Targets:
> > >     x86    - 32-bit X86: Pentium-Pro and above
> > >     x86-64 - 64-bit X86: EM64T and AMD64
> > >
> > > Debian's llc-11 shows me BPF support is built-in.
> > >
> > > I see the breakag approx. 3 hours after the start of my kernel-build -
> > > in the stage "vmlinux".
> > > After 2 faulures in my build (2x 3 hours of build-time) I have still
> > > no finished Linux v5.11-rc3 kernel.
> > > This is a bit frustrating.
> >
> > You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> > Maybe a bug in resolve_btfids due to somehow different ELF format
> > resolve_btfids need to handle?
> >
>
> [ CC Tom ]
>
> OMG no.
>
> 3 hours up to running scripts/link-vmlinux.sh.
>
> In the meantime I have built a LLVM toolchain with BPF support.
>
> $ llc --version
> LLVM (http://llvm.org/):
>  LLVM version 11.0.1
>  Optimized build.
>  Default target: x86_64-unknown-linux-gnu
>  Host CPU: sandybridge
>
>  Registered Targets:
>    bpf    - BPF (host endian)
>    bpfeb  - BPF (big endian)
>    bpfel  - BPF (little endian)

As Yonghong mentioned, you don't need BPF target support in Clang to
build the kernel, so the issue is elsewhere. It's somewhere between
generated DWARF (we've seen multiple bugs in DWARF over time),
pahole's BTF output and resolve_btfids's handling of that BTF. I've
CC'ed Jiri, who can help with resolve_btfids.

Meanwhile, if you can provide SHA from which you built Clang, kernel
config you used, and probably exact invocation of the build you used,
it would help reproduce the issue.

>    x86    - 32-bit X86: Pentium-Pro and above
>    x86-64 - 64-bit X86: EM64T and AMD64
>
> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
> "I ran into this same bug trying to build the Fedora kernel. The
> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
> looks to me like a bug in pahole."
>
> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
> and use it.
>
> Building a new Linux-kernel...
>
> - Sedat -
>
> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553

There are no significant bug fixes between pahole 1.19 and master that
would solve this problem, so let's try to repro this.

>
>
>
> > >
> > > What about doing pre-checks - means before doing a single line of
> > > compilation - to check for:
> > > 1. Required binaries
> > > 2. Required support of whatever feature in compiler, linker, toolchain etc.
> > >
> > > Recently, I fell over depmod binary not found in my PATH - in one of
> > > the last steps (modfinal) of the kernel build.
> > >
> > > Any ideas to improve the situation?
> > > ( ...and please no RTFM, see links below. )
> > >
> > > Thanks.
> > >
> > > Regards,
> > > - Sedat -
> > >
> > >
> > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> > > [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> > > [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> > > [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> > >

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
       [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
@ 2021-01-11 22:31         ` Jiri Olsa
  2021-01-11 22:34           ` Tom Stellard
  2021-01-14 14:13           ` Sedat Dilek
  2021-01-12  0:57         ` Yonghong Song
  1 sibling, 2 replies; 19+ messages in thread
From: Jiri Olsa @ 2021-01-11 22:31 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:

SNIP

> > >
> > > Building a new Linux-kernel...
> > >
> > > - Sedat -
> > >
> > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> >
> > There are no significant bug fixes between pahole 1.19 and master that
> > would solve this problem, so let's try to repro this.
> >
> 
> You are right pahole fom latest Git does not solve the issue.
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>  BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument

hm, is there a .BTF section in vmlinux?

is this working over vmlinux:
 $ bpftool btf dump file ./vmlinux

do you have a verbose build output? I'd think pahole scream first..

jirka


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 22:31         ` Jiri Olsa
@ 2021-01-11 22:34           ` Tom Stellard
  2021-01-12 10:46             ` Jiri Olsa
  2021-01-14 14:13           ` Sedat Dilek
  1 sibling, 1 reply; 19+ messages in thread
From: Tom Stellard @ 2021-01-11 22:34 UTC (permalink / raw)
  To: Jiri Olsa, Sedat Dilek
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list

On 1/11/21 2:31 PM, Jiri Olsa wrote:
> On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> 
> SNIP
> 
>>>>
>>>> Building a new Linux-kernel...
>>>>
>>>> - Sedat -
>>>>
>>>> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
>>>> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
>>>> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
>>>
>>> There are no significant bug fixes between pahole 1.19 and master that
>>> would solve this problem, so let's try to repro this.
>>>
>>
>> You are right pahole fom latest Git does not solve the issue.
>>
>> + info BTFIDS vmlinux
>> + [  != silent_ ]
>> + printf   %-7s %s\n BTFIDS vmlinux
>>   BTFIDS  vmlinux
>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
>> FAILED: load BTF from vmlinux: Invalid argument
> 
> hm, is there a .BTF section in vmlinux?
> 
> is this working over vmlinux:
>   $ bpftool btf dump file ./vmlinux
> 
> do you have a verbose build output? I'd think pahole scream first..
> 

It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This 
is pretty easy for me to reproduce.  I have logs, what other information 
would be helpful?  How about a pahole backtrace?

-Tom

> jirka
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
       [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
  2021-01-11 22:31         ` Jiri Olsa
@ 2021-01-12  0:57         ` Yonghong Song
  2021-01-12  4:09           ` Sedat Dilek
  1 sibling, 1 reply; 19+ messages in thread
From: Yonghong Song @ 2021-01-12  0:57 UTC (permalink / raw)
  To: sedat.dilek, Andrii Nakryiko
  Cc: Jiri Olsa, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list, Tom Stellard



On 1/11/21 1:30 PM, Sedat Dilek wrote:
> On Mon, Jan 11, 2021 at 10:03 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>>
>>> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
>>>>
>>>>
>>>>
>>>> On 1/11/21 4:48 AM, Sedat Dilek wrote:
>>>>> Hi BPF maintainers and Mashiro,
>>>>>
>>>>> Debian started to use CONFIG_DEBUG_INFO_BTF=y.
>>>>>
>>>>> My kernel-build fails like this:
>>>>>
>>>>> + info BTFIDS vmlinux
>>>>> + [  != silent_ ]
>>>>> + printf   %-7s %s\n BTFIDS vmlinux
>>>>>    BTFIDS  vmlinux
>>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
>>>>> FAILED: load BTF from vmlinux: Invalid argument
>>>>>
>>>>> The root cause is my selfmade LLVM toolchain has no BPF support.
>>>>
>>>> linux build should depend on LLVM toolchain unless you use LLVM to build
>>>> kernel.
>>>>
>>>>>
>>>>> $ which llc
>>>>> /home/dileks/src/llvm-toolchain/install/bin/llc
>>>>>
>>>>> $ llc --version
>>>>> LLVM (http://llvm.org/  ):
>>>>>    LLVM version 11.0.1
>>>>>    Optimized build.
>>>>>    Default target: x86_64-unknown-linux-gnu
>>>>>    Host CPU: sandybridge
>>>>>
>>>>>    Registered Targets:
>>>>>      x86    - 32-bit X86: Pentium-Pro and above
>>>>>      x86-64 - 64-bit X86: EM64T and AMD64
>>>>>
>>>>> Debian's llc-11 shows me BPF support is built-in.
>>>>>
>>>>> I see the breakag approx. 3 hours after the start of my kernel-build -
>>>>> in the stage "vmlinux".
>>>>> After 2 faulures in my build (2x 3 hours of build-time) I have still
>>>>> no finished Linux v5.11-rc3 kernel.
>>>>> This is a bit frustrating.
>>>>
>>>> You mean "BTFIDS  vmlinux" takes more than 3 hours here?
>>>> Maybe a bug in resolve_btfids due to somehow different ELF format
>>>> resolve_btfids need to handle?
>>>>
>>>
>>> [ CC Tom ]
>>>
>>> OMG no.
>>>
>>> 3 hours up to running scripts/link-vmlinux.sh.
>>>
>>> In the meantime I have built a LLVM toolchain with BPF support.
>>>
>>> $ llc --version
>>> LLVM (http://llvm.org/ ):
>>>   LLVM version 11.0.1
>>>   Optimized build.
>>>   Default target: x86_64-unknown-linux-gnu
>>>   Host CPU: sandybridge
>>>
>>>   Registered Targets:
>>>     bpf    - BPF (host endian)
>>>     bpfeb  - BPF (big endian)
>>>     bpfel  - BPF (little endian)
>>
>> As Yonghong mentioned, you don't need BPF target support in Clang to
>> build the kernel, so the issue is elsewhere. It's somewhere between
>> generated DWARF (we've seen multiple bugs in DWARF over time),
>> pahole's BTF output and resolve_btfids's handling of that BTF. I've
>> CC'ed Jiri, who can help with resolve_btfids.
>>
>> Meanwhile, if you can provide SHA from which you built Clang, kernel
>> config you used, and probably exact invocation of the build you used,
>> it would help reproduce the issue.
>>
> 
> OK, I see I have here DWARF v5 support patchset applied and enabled.
> 
> Furthermore: I applied latest clang-cfi.
> 
> This is with LLVM v11.0.1 final aka 43ff75f2c3feef64f9d73328230d34dac8832a91.

Did you use llvm to compile kernel? If this is the case, latest pahole 
will segfault. I am using latest trunk llvm. It is possible that 
generated dwarf with llvm is different from generated dwarf with gcc
and pahole did not process it correctly. I did not get time to
debug this though.

> 
> My kernel-config is attached.
> 
> [1] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-1-ndesaulniers@google.com/
> [2] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-2-ndesaulniers@google.com/
> [3] https://github.com/samitolvanen/linux/commits/clang-cfi
> 
>>>     x86    - 32-bit X86: Pentium-43ff75f2c3feef64f9d73328230d34dac8832a91
> Pro and above
>>>     x86-64 - 64-bit X86: EM64T and AMD64
>>>
>>> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
>>> "I ran into this same bug trying to build the Fedora kBROKEN_5-11-rc3-CONFIG_DEBUG_INFO_BTF-y-FAILED-load-BTF-from-vmlinux.txt
> ernel. The
>>> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
>>> looks to me like a bug in pahole."
>>>
>>> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
>>> and use it.
>>>
>>> Building a new Linux-kernel...
>>>
>>> - Sedat -
>>>
>>> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
>>> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
>>> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
>>
>> There are no significant bug fixes between pahole 1.19 and master that
>> would solve this problem, so let's try to repro this.
>>
> 
> You are right pahole fom latest Git does not solve the issue.
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>   BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument
> 
> - Sedat -
> 
>>>
>>>
>>>
>>>>>
>>>>> What about doing pre-checks - means before doing a single line of
>>>>> compilation - to check for:
>>>>> 1. Required binaries
>>>>> 2. Required support of whatever feature in compiler, linker, toolchain etc.
>>>>>
>>>>> Recently, I fell over depmod binary not found in my PATH - in one of
>>>>> the last steps (modfinal) of the kernel build.
>>>>>
>>>>> Any ideas to improve the situation?
>>>>> ( ...and please no RTFM, see links below. )
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Regards,
>>>>> - Sedat -
>>>>>
>>>>>
>>>>> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
>>>>> [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
>>>>> [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
>>>>> [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
>>>>>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12  0:57         ` Yonghong Song
@ 2021-01-12  4:09           ` Sedat Dilek
  0 siblings, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12  4:09 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Andrii Nakryiko, Jiri Olsa, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend,
	KP Singh, Masahiro Yamada, bpf, Linux Kbuild mailing list,
	Tom Stellard

On Tue, Jan 12, 2021 at 1:57 AM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 1/11/21 1:30 PM, Sedat Dilek wrote:
> > On Mon, Jan 11, 2021 at 10:03 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> >>
> >> On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >>>
> >>> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 1/11/21 4:48 AM, Sedat Dilek wrote:
> >>>>> Hi BPF maintainers and Mashiro,
> >>>>>
> >>>>> Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> >>>>>
> >>>>> My kernel-build fails like this:
> >>>>>
> >>>>> + info BTFIDS vmlinux
> >>>>> + [  != silent_ ]
> >>>>> + printf   %-7s %s\n BTFIDS vmlinux
> >>>>>    BTFIDS  vmlinux
> >>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> >>>>> FAILED: load BTF from vmlinux: Invalid argument
> >>>>>
> >>>>> The root cause is my selfmade LLVM toolchain has no BPF support.
> >>>>
> >>>> linux build should depend on LLVM toolchain unless you use LLVM to build
> >>>> kernel.
> >>>>
> >>>>>
> >>>>> $ which llc
> >>>>> /home/dileks/src/llvm-toolchain/install/bin/llc
> >>>>>
> >>>>> $ llc --version
> >>>>> LLVM (http://llvm.org/  ):
> >>>>>    LLVM version 11.0.1
> >>>>>    Optimized build.
> >>>>>    Default target: x86_64-unknown-linux-gnu
> >>>>>    Host CPU: sandybridge
> >>>>>
> >>>>>    Registered Targets:
> >>>>>      x86    - 32-bit X86: Pentium-Pro and above
> >>>>>      x86-64 - 64-bit X86: EM64T and AMD64
> >>>>>
> >>>>> Debian's llc-11 shows me BPF support is built-in.
> >>>>>
> >>>>> I see the breakag approx. 3 hours after the start of my kernel-build -
> >>>>> in the stage "vmlinux".
> >>>>> After 2 faulures in my build (2x 3 hours of build-time) I have still
> >>>>> no finished Linux v5.11-rc3 kernel.
> >>>>> This is a bit frustrating.
> >>>>
> >>>> You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> >>>> Maybe a bug in resolve_btfids due to somehow different ELF format
> >>>> resolve_btfids need to handle?
> >>>>
> >>>
> >>> [ CC Tom ]
> >>>
> >>> OMG no.
> >>>
> >>> 3 hours up to running scripts/link-vmlinux.sh.
> >>>
> >>> In the meantime I have built a LLVM toolchain with BPF support.
> >>>
> >>> $ llc --version
> >>> LLVM (http://llvm.org/ ):
> >>>   LLVM version 11.0.1
> >>>   Optimized build.
> >>>   Default target: x86_64-unknown-linux-gnu
> >>>   Host CPU: sandybridge
> >>>
> >>>   Registered Targets:
> >>>     bpf    - BPF (host endian)
> >>>     bpfeb  - BPF (big endian)
> >>>     bpfel  - BPF (little endian)
> >>
> >> As Yonghong mentioned, you don't need BPF target support in Clang to
> >> build the kernel, so the issue is elsewhere. It's somewhere between
> >> generated DWARF (we've seen multiple bugs in DWARF over time),
> >> pahole's BTF output and resolve_btfids's handling of that BTF. I've
> >> CC'ed Jiri, who can help with resolve_btfids.
> >>
> >> Meanwhile, if you can provide SHA from which you built Clang, kernel
> >> config you used, and probably exact invocation of the build you used,
> >> it would help reproduce the issue.
> >>
> >
> > OK, I see I have here DWARF v5 support patchset applied and enabled.
> >
> > Furthermore: I applied latest clang-cfi.
> >
> > This is with LLVM v11.0.1 final aka 43ff75f2c3feef64f9d73328230d34dac8832a91.
>
> Did you use llvm to compile kernel? If this is the case, latest pahole
> will segfault. I am using latest trunk llvm. It is possible that
> generated dwarf with llvm is different from generated dwarf with gcc
> and pahole did not process it correctly. I did not get time to
> debug this though.
>

Yes, I used LLVM/Clang to compile the kernel.

- Sedat -

> >
> > My kernel-config is attached.
> >
> > [1] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-1-ndesaulniers@google.com/
> > [2] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-2-ndesaulniers@google.com/
> > [3] https://github.com/samitolvanen/linux/commits/clang-cfi
> >
> >>>     x86    - 32-bit X86: Pentium-43ff75f2c3feef64f9d73328230d34dac8832a91
> > Pro and above
> >>>     x86-64 - 64-bit X86: EM64T and AMD64
> >>>
> >>> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
> >>> "I ran into this same bug trying to build the Fedora kBROKEN_5-11-rc3-CONFIG_DEBUG_INFO_BTF-y-FAILED-load-BTF-from-vmlinux.txt
> > ernel. The
> >>> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
> >>> looks to me like a bug in pahole."
> >>>
> >>> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
> >>> and use it.
> >>>
> >>> Building a new Linux-kernel...
> >>>
> >>> - Sedat -
> >>>
> >>> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> >>> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> >>> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> >>
> >> There are no significant bug fixes between pahole 1.19 and master that
> >> would solve this problem, so let's try to repro this.
> >>
> >
> > You are right pahole fom latest Git does not solve the issue.
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >   BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
> >
> > - Sedat -
> >
> >>>
> >>>
> >>>
> >>>>>
> >>>>> What about doing pre-checks - means before doing a single line of
> >>>>> compilation - to check for:
> >>>>> 1. Required binaries
> >>>>> 2. Required support of whatever feature in compiler, linker, toolchain etc.
> >>>>>
> >>>>> Recently, I fell over depmod binary not found in my PATH - in one of
> >>>>> the last steps (modfinal) of the kernel build.
> >>>>>
> >>>>> Any ideas to improve the situation?
> >>>>> ( ...and please no RTFM, see links below. )
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> Regards,
> >>>>> - Sedat -
> >>>>>
> >>>>>
> >>>>> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> >>>>> [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> >>>>> [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> >>>>> [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> >>>>>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 22:34           ` Tom Stellard
@ 2021-01-12 10:46             ` Jiri Olsa
  2021-01-12 13:10               ` Jiri Olsa
  0 siblings, 1 reply; 19+ messages in thread
From: Jiri Olsa @ 2021-01-12 10:46 UTC (permalink / raw)
  To: Tom Stellard
  Cc: Sedat Dilek, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > 
> > SNIP
> > 
> > > > > 
> > > > > Building a new Linux-kernel...
> > > > > 
> > > > > - Sedat -
> > > > > 
> > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > 
> > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > would solve this problem, so let's try to repro this.
> > > > 
> > > 
> > > You are right pahole fom latest Git does not solve the issue.
> > > 
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >   BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> > 
> > hm, is there a .BTF section in vmlinux?
> > 
> > is this working over vmlinux:
> >   $ bpftool btf dump file ./vmlinux
> > 
> > do you have a verbose build output? I'd think pahole scream first..
> > 
> 
> It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> pretty easy for me to reproduce.  I have logs, what other information would
> be helpful?  How about a pahole backtrace?

that'd be great.. I'll try to reproduce, but with the latest clang
it will take me some time

jirka

> 
> -Tom
> 
> > jirka
> > 
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 10:46             ` Jiri Olsa
@ 2021-01-12 13:10               ` Jiri Olsa
  2021-01-12 16:14                 ` Sedat Dilek
  0 siblings, 1 reply; 19+ messages in thread
From: Jiri Olsa @ 2021-01-12 13:10 UTC (permalink / raw)
  To: Tom Stellard
  Cc: Sedat Dilek, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > 
> > > SNIP
> > > 
> > > > > > 
> > > > > > Building a new Linux-kernel...
> > > > > > 
> > > > > > - Sedat -
> > > > > > 
> > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > 
> > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > would solve this problem, so let's try to repro this.
> > > > > 
> > > > 
> > > > You are right pahole fom latest Git does not solve the issue.
> > > > 
> > > > + info BTFIDS vmlinux
> > > > + [  != silent_ ]
> > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > >   BTFIDS  vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > > 
> > > hm, is there a .BTF section in vmlinux?
> > > 
> > > is this working over vmlinux:
> > >   $ bpftool btf dump file ./vmlinux
> > > 
> > > do you have a verbose build output? I'd think pahole scream first..
> > > 
> > 
> > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > pretty easy for me to reproduce.  I have logs, what other information would
> > be helpful?  How about a pahole backtrace?
> 
> that'd be great.. I'll try to reproduce, but with the latest clang
> it will take me some time

reproduced, attached pahole patch fixes it for me,

looks like gcc never left function without name,
which does not seem to be the case for clang

I'll send full patch later today

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index c40f059580da..781fb35a2646 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -70,6 +70,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
 		return 0;
 	if (!elf_sym__value(sym))
 		return 0;
+	if (!elf_sym__name(sym, btfe->symtab))
+		return 0;
 
 	if (functions_cnt == functions_alloc) {
 		functions_alloc = max(1000, functions_alloc * 3 / 2);
@@ -620,9 +622,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
 		if (!has_arg_names(cu, &fn->proto))
 			continue;
 		if (functions_cnt) {
-			struct elf_function *func;
+			const char *name = function__name(fn, cu);
+			struct elf_function *func = NULL;
 
-			func = find_function(btfe, function__name(fn, cu));
+			if (name)
+				func = find_function(btfe, name);
 			if (!func || func->generated)
 				continue;
 			func->generated = true;


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 13:10               ` Jiri Olsa
@ 2021-01-12 16:14                 ` Sedat Dilek
  2021-01-12 16:21                   ` Jiri Olsa
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 16:14 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 2:10 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> > On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > >
> > > > SNIP
> > > >
> > > > > > >
> > > > > > > Building a new Linux-kernel...
> > > > > > >
> > > > > > > - Sedat -
> > > > > > >
> > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > >
> > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > would solve this problem, so let's try to repro this.
> > > > > >
> > > > >
> > > > > You are right pahole fom latest Git does not solve the issue.
> > > > >
> > > > > + info BTFIDS vmlinux
> > > > > + [  != silent_ ]
> > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > >   BTFIDS  vmlinux
> > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > >
> > > > hm, is there a .BTF section in vmlinux?
> > > >
> > > > is this working over vmlinux:
> > > >   $ bpftool btf dump file ./vmlinux
> > > >
> > > > do you have a verbose build output? I'd think pahole scream first..
> > > >
> > >
> > > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > > pretty easy for me to reproduce.  I have logs, what other information would
> > > be helpful?  How about a pahole backtrace?
> >
> > that'd be great.. I'll try to reproduce, but with the latest clang
> > it will take me some time
>
> reproduced, attached pahole patch fixes it for me,
>
> looks like gcc never left function without name,
> which does not seem to be the case for clang
>
> I'll send full patch later today
>

Thanks for the diff.

Unfortunately, it does not apply on latest pahole git.

$ git describe
v1.19-7-gb688e3597060

- Sedat -

> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index c40f059580da..781fb35a2646 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -70,6 +70,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
>                 return 0;
>         if (!elf_sym__value(sym))
>                 return 0;
> +       if (!elf_sym__name(sym, btfe->symtab))
> +               return 0;
>
>         if (functions_cnt == functions_alloc) {
>                 functions_alloc = max(1000, functions_alloc * 3 / 2);
> @@ -620,9 +622,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
>                 if (!has_arg_names(cu, &fn->proto))
>                         continue;
>                 if (functions_cnt) {
> -                       struct elf_function *func;
> +                       const char *name = function__name(fn, cu);
> +                       struct elf_function *func = NULL;
>
> -                       func = find_function(btfe, function__name(fn, cu));
> +                       if (name)
> +                               func = find_function(btfe, name);
>                         if (!func || func->generated)
>                                 continue;
>                         func->generated = true;
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 16:14                 ` Sedat Dilek
@ 2021-01-12 16:21                   ` Jiri Olsa
  2021-01-12 17:18                     ` Sedat Dilek
  2021-01-12 20:47                     ` Sedat Dilek
  0 siblings, 2 replies; 19+ messages in thread
From: Jiri Olsa @ 2021-01-12 16:21 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 05:14:42PM +0100, Sedat Dilek wrote:
> On Tue, Jan 12, 2021 at 2:10 PM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> > > On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > > > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > > >
> > > > > SNIP
> > > > >
> > > > > > > >
> > > > > > > > Building a new Linux-kernel...
> > > > > > > >
> > > > > > > > - Sedat -
> > > > > > > >
> > > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > > >
> > > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > > would solve this problem, so let's try to repro this.
> > > > > > >
> > > > > >
> > > > > > You are right pahole fom latest Git does not solve the issue.
> > > > > >
> > > > > > + info BTFIDS vmlinux
> > > > > > + [  != silent_ ]
> > > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > > >   BTFIDS  vmlinux
> > > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > >
> > > > > hm, is there a .BTF section in vmlinux?
> > > > >
> > > > > is this working over vmlinux:
> > > > >   $ bpftool btf dump file ./vmlinux
> > > > >
> > > > > do you have a verbose build output? I'd think pahole scream first..
> > > > >
> > > >
> > > > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > > > pretty easy for me to reproduce.  I have logs, what other information would
> > > > be helpful?  How about a pahole backtrace?
> > >
> > > that'd be great.. I'll try to reproduce, but with the latest clang
> > > it will take me some time
> >
> > reproduced, attached pahole patch fixes it for me,
> >
> > looks like gcc never left function without name,
> > which does not seem to be the case for clang
> >
> > I'll send full patch later today
> >
> 
> Thanks for the diff.
> 
> Unfortunately, it does not apply on latest pahole git.
> 
> $ git describe
> v1.19-7-gb688e3597060

sry wrong master.. how about this one

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index 333973054b61..17f7a14f2ef0 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -72,6 +72,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
 
 	if (elf_sym__type(sym) != STT_FUNC)
 		return 0;
+	if (!elf_sym__name(sym, btfe->symtab))
+		return 0;
 
 	if (functions_cnt == functions_alloc) {
 		functions_alloc = max(1000, functions_alloc * 3 / 2);
@@ -730,9 +732,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
 		if (!has_arg_names(cu, &fn->proto))
 			continue;
 		if (functions_cnt) {
-			struct elf_function *func;
+			const char *name = function__name(fn, cu);
+			struct elf_function *func = NULL;
 
-			func = find_function(btfe, function__name(fn, cu));
+			if (name)
+				func = find_function(btfe, name);
 			if (!func || func->generated)
 				continue;
 			func->generated = true;
-- 
2.26.2


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 16:21                   ` Jiri Olsa
@ 2021-01-12 17:18                     ` Sedat Dilek
  2021-01-12 20:47                     ` Sedat Dilek
  1 sibling, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 17:18 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 5:22 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Jan 12, 2021 at 05:14:42PM +0100, Sedat Dilek wrote:
> > On Tue, Jan 12, 2021 at 2:10 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> > > > On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > > > > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > > > >
> > > > > > SNIP
> > > > > >
> > > > > > > > >
> > > > > > > > > Building a new Linux-kernel...
> > > > > > > > >
> > > > > > > > > - Sedat -
> > > > > > > > >
> > > > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > > > >
> > > > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > > > would solve this problem, so let's try to repro this.
> > > > > > > >
> > > > > > >
> > > > > > > You are right pahole fom latest Git does not solve the issue.
> > > > > > >
> > > > > > > + info BTFIDS vmlinux
> > > > > > > + [  != silent_ ]
> > > > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > > > >   BTFIDS  vmlinux
> > > > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > > >
> > > > > > hm, is there a .BTF section in vmlinux?
> > > > > >
> > > > > > is this working over vmlinux:
> > > > > >   $ bpftool btf dump file ./vmlinux
> > > > > >
> > > > > > do you have a verbose build output? I'd think pahole scream first..
> > > > > >
> > > > >
> > > > > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > > > > pretty easy for me to reproduce.  I have logs, what other information would
> > > > > be helpful?  How about a pahole backtrace?
> > > >
> > > > that'd be great.. I'll try to reproduce, but with the latest clang
> > > > it will take me some time
> > >
> > > reproduced, attached pahole patch fixes it for me,
> > >
> > > looks like gcc never left function without name,
> > > which does not seem to be the case for clang
> > >
> > > I'll send full patch later today
> > >
> >
> > Thanks for the diff.
> >
> > Unfortunately, it does not apply on latest pahole git.
> >
> > $ git describe
> > v1.19-7-gb688e3597060
>
> sry wrong master.. how about this one
>

I have applied the previous diff on top of tagsv1.19.

Thanks for the follow-up.

Please CC me on the patch you wanted to send later.

- Sedat -

> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index 333973054b61..17f7a14f2ef0 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -72,6 +72,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
>
>         if (elf_sym__type(sym) != STT_FUNC)
>                 return 0;
> +       if (!elf_sym__name(sym, btfe->symtab))
> +               return 0;
>
>         if (functions_cnt == functions_alloc) {
>                 functions_alloc = max(1000, functions_alloc * 3 / 2);
> @@ -730,9 +732,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
>                 if (!has_arg_names(cu, &fn->proto))
>                         continue;
>                 if (functions_cnt) {
> -                       struct elf_function *func;
> +                       const char *name = function__name(fn, cu);
> +                       struct elf_function *func = NULL;
>
> -                       func = find_function(btfe, function__name(fn, cu));
> +                       if (name)
> +                               func = find_function(btfe, name);
>                         if (!func || func->generated)
>                                 continue;
>                         func->generated = true;
> --
> 2.26.2
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 16:21                   ` Jiri Olsa
  2021-01-12 17:18                     ` Sedat Dilek
@ 2021-01-12 20:47                     ` Sedat Dilek
  2021-01-12 21:36                       ` Sedat Dilek
  1 sibling, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 20:47 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

Is it possible that someone tests with Nick Desaulniers's DWARF v5 patchset?

With b4 tool installed:

link="https://lore.kernel.org/r/CAKwvOdkZEiHK01OD420USb0j=F0LcrnRbauv9Yw26tu-GRbYkg@mail.gmail.com"
b4 -d am $link

...should give you a file which you can directly apply with git tool.

File: v3_20201203_ndesaulniers_kbuild_make_dwarf_version_a_choice.mbx

As my selfmade LLVM toolchain seems to be no good choice I jumped to gcc-10.

I see gazillions of DW_AT_ and DW_TAG_INVALID (0x48) warnings:

die__process_inline_expansion: DW_TAG_INVALID (0x48) @ <0x1c2a25e> not handled!

This might be due to DWARF version 5 patchset.
I did not check DWARF v2 (default) or DWARF v4 choices.

Beyond this, I noticed:

$ git grep LLVM_OBJCOPY scripts/ tools/
scripts/Makefile.modfinal:              LLVM_OBJCOPY=$(OBJCOPY)
$(PAHOLE) -J --btf_base vmlinux $@; \
scripts/link-vmlinux.sh:        LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
tools/testing/selftests/bpf/Makefile:LLVM_OBJCOPY       ?= llvm-objcopy

When only passing gcc-10 as compiler and ld.bfd as linker -
LLVM_OBJCOPY=objcopy is passed.
Means objcopy from GNU/binutils is used.
As far as I understand the code you want here llvm-objcopy
(scripts/Makefile.modfinal)?

I did a next tryout with gcc-10 and LLVM=1 means use all equivalent
LLVM tools to GNU/binutils.
Means use llvm-objcopy, llvm-nm, llvm-ar, llvm-objdump, etc.
Please see the link below under [1].

En plus, I tried pahole with Jirka "Convulted" Olsa (JCO) fix by passing...
   PAHOLE=/opt/pahole/bin/pahole
...to my make line.

Again I see that DW_TAG_INVALID (0x48) warnings.
Can someone enlighten me what that means?

Finally, with all that warnings observed I could "successfully"
generate Linux-kernel Debian packages.
I do NOT install software I do NOT trust - not booted - NoNoNo...

Thanks.

In the dark,
- Sedat -

[1] https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 20:47                     ` Sedat Dilek
@ 2021-01-12 21:36                       ` Sedat Dilek
  0 siblings, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 21:36 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 9:47 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:

> En plus, I tried pahole with Jirka "Convulted" Olsa (JCO) fix by passing...
>    PAHOLE=/opt/pahole/bin/pahole
> ...to my make line.
>

Grrr, my selfmade pahole misses that patch.

How can I re-invoke make ... with new really fixed pahole version?

Fallen into a (pa)hole,
- Sedat -

P.S.: Download Jiri's patch

link="https://lore.kernel.org/bpf/20210112194724.GB1291051@krava/T/#t"
b4 -d am $link

- EOT -

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 22:31         ` Jiri Olsa
  2021-01-11 22:34           ` Tom Stellard
@ 2021-01-14 14:13           ` Sedat Dilek
  2021-01-14 14:24             ` Jiri Olsa
  1 sibling, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-14 14:13 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
>
> SNIP
>
> > > >
> > > > Building a new Linux-kernel...
> > > >
> > > > - Sedat -
> > > >
> > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > >
> > > There are no significant bug fixes between pahole 1.19 and master that
> > > would solve this problem, so let's try to repro this.
> > >
> >
> > You are right pahole fom latest Git does not solve the issue.
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >  BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
>
> hm, is there a .BTF section in vmlinux?
>
> is this working over vmlinux:
>  $ bpftool btf dump file ./vmlinux
>

I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.

The generated vmlinux file is cleaned on failure.

+ info BTFIDS vmlinux
+ [  != silent_ ]
+ printf   %-7s %s\n BTFIDS vmlinux
 BTFIDS  vmlinux
+ ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
FAILED: load BTF from vmlinux: Invalid argument
+ on_exit
+ [ 255 -ne 0 ]
+ cleanup
+ rm -f .btf.vmlinux.bin.o
+ rm -f .tmp_System.map
+ rm -f .tmp_initcalls.lds
+ rm -f .tmp_symversions.lds
+ rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
.tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
.tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
2.o
+ rm -f System.map
+ rm -f vmlinux
+ rm -f vmlinux.o

Dunno, how to suppress this.

- Sedat -

> do you have a verbose build output? I'd think pahole scream first..
>
> jirka
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-14 14:13           ` Sedat Dilek
@ 2021-01-14 14:24             ` Jiri Olsa
  2021-01-14 14:29               ` Sedat Dilek
  0 siblings, 1 reply; 19+ messages in thread
From: Jiri Olsa @ 2021-01-14 14:24 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Thu, Jan 14, 2021 at 03:13:18PM +0100, Sedat Dilek wrote:
> On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> >
> > SNIP
> >
> > > > >
> > > > > Building a new Linux-kernel...
> > > > >
> > > > > - Sedat -
> > > > >
> > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > >
> > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > would solve this problem, so let's try to repro this.
> > > >
> > >
> > > You are right pahole fom latest Git does not solve the issue.
> > >
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >  BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> >
> > hm, is there a .BTF section in vmlinux?
> >
> > is this working over vmlinux:
> >  $ bpftool btf dump file ./vmlinux
> >
> 
> I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.
> 
> The generated vmlinux file is cleaned on failure.
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>  BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument

did pahole generated the .BTF section? earlier in the log

jirka


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-14 14:24             ` Jiri Olsa
@ 2021-01-14 14:29               ` Sedat Dilek
       [not found]                 ` <CA+icZUXp5gkxCr+mybQ8uO3xx+rVmmRSnN7Dfx0Xmr+VQWyZAA@mail.gmail.com>
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-14 14:29 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Thu, Jan 14, 2021 at 3:24 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Thu, Jan 14, 2021 at 03:13:18PM +0100, Sedat Dilek wrote:
> > On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > >
> > > SNIP
> > >
> > > > > >
> > > > > > Building a new Linux-kernel...
> > > > > >
> > > > > > - Sedat -
> > > > > >
> > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > >
> > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > would solve this problem, so let's try to repro this.
> > > > >
> > > >
> > > > You are right pahole fom latest Git does not solve the issue.
> > > >
> > > > + info BTFIDS vmlinux
> > > > + [  != silent_ ]
> > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > >  BTFIDS  vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > hm, is there a .BTF section in vmlinux?
> > >
> > > is this working over vmlinux:
> > >  $ bpftool btf dump file ./vmlinux
> > >
> >
> > I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.
> >
> > The generated vmlinux file is cleaned on failure.
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >  BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
>
> did pahole generated the .BTF section? earlier in the log
>
> jirka
>

Cannot say.

I did:

$ git diff
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 2d0b28758aa5..c6e679d08bbe 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -401,12 +401,6 @@ fi

vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}

-# fill in BTF IDs
-if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
-       info BTFIDS vmlinux
-       ${RESOLVE_BTFIDS} vmlinux
-fi
-
if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
       info SORTTAB vmlinux
       if ! sorttable vmlinux; then

- Sedat -

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
       [not found]                 ` <CA+icZUXp5gkxCr+mybQ8uO3xx+rVmmRSnN7Dfx0Xmr+VQWyZAA@mail.gmail.com>
@ 2021-01-14 15:13                   ` Sedat Dilek
  0 siblings, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-14 15:13 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard


[-- Attachment #1: Type: text/plain, Size: 3509 bytes --]

On Thu, Jan 14, 2021 at 3:50 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Thu, Jan 14, 2021 at 3:29 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Thu, Jan 14, 2021 at 3:24 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Thu, Jan 14, 2021 at 03:13:18PM +0100, Sedat Dilek wrote:
> > > > On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > > > >
> > > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > > >
> > > > > SNIP
> > > > >
> > > > > > > >
> > > > > > > > Building a new Linux-kernel...
> > > > > > > >
> > > > > > > > - Sedat -
> > > > > > > >
> > > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > > >
> > > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > > would solve this problem, so let's try to repro this.
> > > > > > >
> > > > > >
> > > > > > You are right pahole fom latest Git does not solve the issue.
> > > > > >
> > > > > > + info BTFIDS vmlinux
> > > > > > + [  != silent_ ]
> > > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > > >  BTFIDS  vmlinux
> > > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > >
> > > > > hm, is there a .BTF section in vmlinux?
> > > > >
> > > > > is this working over vmlinux:
> > > > >  $ bpftool btf dump file ./vmlinux
> > > > >
> > > >
> > > > I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.
> > > >
> > > > The generated vmlinux file is cleaned on failure.
> > > >
> > > > + info BTFIDS vmlinux
> > > > + [  != silent_ ]
> > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > >  BTFIDS  vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > did pahole generated the .BTF section? earlier in the log
> > >
> > > jirka
> > >
> >
> > Cannot say.
> >
> > I did:
> >
> > $ git diff
> > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> > index 2d0b28758aa5..c6e679d08bbe 100755
> > --- a/scripts/link-vmlinux.sh
> > +++ b/scripts/link-vmlinux.sh
> > @@ -401,12 +401,6 @@ fi
> >
> > vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
> >
> > -# fill in BTF IDs
> > -if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
> > -       info BTFIDS vmlinux
> > -       ${RESOLVE_BTFIDS} vmlinux
> > -fi
> > -
> > if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
> >        info SORTTAB vmlinux
> >        if ! sorttable vmlinux; then
> >
>
> Before...
>
> Encountered error while encoding BTF.
> + llvm-objcopy --only-section=.BTF --set-section-flags
> .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
>
> ...I see lots of errors.
>
> I have cut the relevant stuff and placed it in the attachment.
>
> With freshly installed bpftool:
>
> $ /usr/sbin/bpftool btf dump file ./vmlinux
> Error: failed to load BTF from ./vmlinux: Invalid argument
>

This is with CONFIG_DEBUG_INFO_DWARF4=y.
( See attached patch with cover-letter - applied here on top of Linux
v5.11-rc3+).

Concerning this I am looking into [1]:

die__process_unit: DW_TAG_label (0xa) @ <0xa7> not handled!
die__process_unit: tag not supported 0xa (label)!

- Sedat -

[1] http://www.dwarfstd.org/doc/DWARF4.pdf > 3.5 LABEL ENTRIES

[-- Attachment #2: v4_20210112_ndesaulniers_kbuild_dwarf_v5_support.cover --]
[-- Type: application/octet-stream, Size: 3093 bytes --]

[-- Attachment #3: v4_20210112_ndesaulniers_kbuild_dwarf_v5_support.mbx --]
[-- Type: application/octet-stream, Size: 12905 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, back to index

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-11 12:48 Check pahole availibity and BPF support of toolchain before starting a Linux kernel build Sedat Dilek
2021-01-11 16:05 ` Yonghong Song
2021-01-11 17:56   ` Sedat Dilek
2021-01-11 21:03     ` Andrii Nakryiko
     [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
2021-01-11 22:31         ` Jiri Olsa
2021-01-11 22:34           ` Tom Stellard
2021-01-12 10:46             ` Jiri Olsa
2021-01-12 13:10               ` Jiri Olsa
2021-01-12 16:14                 ` Sedat Dilek
2021-01-12 16:21                   ` Jiri Olsa
2021-01-12 17:18                     ` Sedat Dilek
2021-01-12 20:47                     ` Sedat Dilek
2021-01-12 21:36                       ` Sedat Dilek
2021-01-14 14:13           ` Sedat Dilek
2021-01-14 14:24             ` Jiri Olsa
2021-01-14 14:29               ` Sedat Dilek
     [not found]                 ` <CA+icZUXp5gkxCr+mybQ8uO3xx+rVmmRSnN7Dfx0Xmr+VQWyZAA@mail.gmail.com>
2021-01-14 15:13                   ` Sedat Dilek
2021-01-12  0:57         ` Yonghong Song
2021-01-12  4:09           ` Sedat Dilek

BPF Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/bpf/0 bpf/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 bpf bpf/ https://lore.kernel.org/bpf \
		bpf@vger.kernel.org
	public-inbox-index bpf

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.bpf


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git