bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
@ 2021-01-11 12:48 Sedat Dilek
  2021-01-11 16:05 ` Yonghong Song
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-11 12:48 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Masahiro Yamada
  Cc: bpf, linux-kbuild

Hi BPF maintainers and Mashiro,

Debian started to use CONFIG_DEBUG_INFO_BTF=y.

My kernel-build fails like this:

+ info BTFIDS vmlinux
+ [  != silent_ ]
+ printf   %-7s %s\n BTFIDS vmlinux
 BTFIDS  vmlinux
+ ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
FAILED: load BTF from vmlinux: Invalid argument

The root cause is my selfmade LLVM toolchain has no BPF support.

$ which llc
/home/dileks/src/llvm-toolchain/install/bin/llc

$ llc --version
LLVM (http://llvm.org/):
 LLVM version 11.0.1
 Optimized build.
 Default target: x86_64-unknown-linux-gnu
 Host CPU: sandybridge

 Registered Targets:
   x86    - 32-bit X86: Pentium-Pro and above
   x86-64 - 64-bit X86: EM64T and AMD64

Debian's llc-11 shows me BPF support is built-in.

I see the breakag approx. 3 hours after the start of my kernel-build -
in the stage "vmlinux".
After 2 faulures in my build (2x 3 hours of build-time) I have still
no finished Linux v5.11-rc3 kernel.
This is a bit frustrating.

What about doing pre-checks - means before doing a single line of
compilation - to check for:
1. Required binaries
2. Required support of whatever feature in compiler, linker, toolchain etc.

Recently, I fell over depmod binary not found in my PATH - in one of
the last steps (modfinal) of the kernel build.

Any ideas to improve the situation?
( ...and please no RTFM, see links below. )

Thanks.

Regards,
- Sedat -


[0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
[1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
[2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
[3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 12:48 Check pahole availibity and BPF support of toolchain before starting a Linux kernel build Sedat Dilek
@ 2021-01-11 16:05 ` Yonghong Song
  2021-01-11 17:56   ` Sedat Dilek
  0 siblings, 1 reply; 19+ messages in thread
From: Yonghong Song @ 2021-01-11 16:05 UTC (permalink / raw)
  To: sedat.dilek, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend,
	KP Singh, Masahiro Yamada
  Cc: bpf, linux-kbuild



On 1/11/21 4:48 AM, Sedat Dilek wrote:
> Hi BPF maintainers and Mashiro,
> 
> Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> 
> My kernel-build fails like this:
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>   BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument
> 
> The root cause is my selfmade LLVM toolchain has no BPF support.

linux build should depend on LLVM toolchain unless you use LLVM to build 
kernel.

> 
> $ which llc
> /home/dileks/src/llvm-toolchain/install/bin/llc
> 
> $ llc --version
> LLVM (http://llvm.org/ ):
>   LLVM version 11.0.1
>   Optimized build.
>   Default target: x86_64-unknown-linux-gnu
>   Host CPU: sandybridge
> 
>   Registered Targets:
>     x86    - 32-bit X86: Pentium-Pro and above
>     x86-64 - 64-bit X86: EM64T and AMD64
> 
> Debian's llc-11 shows me BPF support is built-in.
> 
> I see the breakag approx. 3 hours after the start of my kernel-build -
> in the stage "vmlinux".
> After 2 faulures in my build (2x 3 hours of build-time) I have still
> no finished Linux v5.11-rc3 kernel.
> This is a bit frustrating.

You mean "BTFIDS  vmlinux" takes more than 3 hours here?
Maybe a bug in resolve_btfids due to somehow different ELF format
resolve_btfids need to handle?

> 
> What about doing pre-checks - means before doing a single line of
> compilation - to check for:
> 1. Required binaries
> 2. Required support of whatever feature in compiler, linker, toolchain etc.
> 
> Recently, I fell over depmod binary not found in my PATH - in one of
> the last steps (modfinal) of the kernel build.
> 
> Any ideas to improve the situation?
> ( ...and please no RTFM, see links below. )
> 
> Thanks.
> 
> Regards,
> - Sedat -
> 
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> 

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 16:05 ` Yonghong Song
@ 2021-01-11 17:56   ` Sedat Dilek
  2021-01-11 21:03     ` Andrii Nakryiko
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-11 17:56 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, linux-kbuild, Tom Stellard

On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 1/11/21 4:48 AM, Sedat Dilek wrote:
> > Hi BPF maintainers and Mashiro,
> >
> > Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> >
> > My kernel-build fails like this:
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >   BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
> >
> > The root cause is my selfmade LLVM toolchain has no BPF support.
>
> linux build should depend on LLVM toolchain unless you use LLVM to build
> kernel.
>
> >
> > $ which llc
> > /home/dileks/src/llvm-toolchain/install/bin/llc
> >
> > $ llc --version
> > LLVM (http://llvm.org/ ):
> >   LLVM version 11.0.1
> >   Optimized build.
> >   Default target: x86_64-unknown-linux-gnu
> >   Host CPU: sandybridge
> >
> >   Registered Targets:
> >     x86    - 32-bit X86: Pentium-Pro and above
> >     x86-64 - 64-bit X86: EM64T and AMD64
> >
> > Debian's llc-11 shows me BPF support is built-in.
> >
> > I see the breakag approx. 3 hours after the start of my kernel-build -
> > in the stage "vmlinux".
> > After 2 faulures in my build (2x 3 hours of build-time) I have still
> > no finished Linux v5.11-rc3 kernel.
> > This is a bit frustrating.
>
> You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> Maybe a bug in resolve_btfids due to somehow different ELF format
> resolve_btfids need to handle?
>

[ CC Tom ]

OMG no.

3 hours up to running scripts/link-vmlinux.sh.

In the meantime I have built a LLVM toolchain with BPF support.

$ llc --version
LLVM (http://llvm.org/):
 LLVM version 11.0.1
 Optimized build.
 Default target: x86_64-unknown-linux-gnu
 Host CPU: sandybridge

 Registered Targets:
   bpf    - BPF (host endian)
   bpfeb  - BPF (big endian)
   bpfel  - BPF (little endian)
   x86    - 32-bit X86: Pentium-Pro and above
   x86-64 - 64-bit X86: EM64T and AMD64

Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
"I ran into this same bug trying to build the Fedora kernel. The
problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
looks to me like a bug in pahole."

pahole ToT (post v1.19) offers some BTF fixes - I have manually build
and use it.

Building a new Linux-kernel...

- Sedat -

[1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
[2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
[3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553



> >
> > What about doing pre-checks - means before doing a single line of
> > compilation - to check for:
> > 1. Required binaries
> > 2. Required support of whatever feature in compiler, linker, toolchain etc.
> >
> > Recently, I fell over depmod binary not found in my PATH - in one of
> > the last steps (modfinal) of the kernel build.
> >
> > Any ideas to improve the situation?
> > ( ...and please no RTFM, see links below. )
> >
> > Thanks.
> >
> > Regards,
> > - Sedat -
> >
> >
> > [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> > [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> > [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> > [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> >

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 17:56   ` Sedat Dilek
@ 2021-01-11 21:03     ` Andrii Nakryiko
       [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
  0 siblings, 1 reply; 19+ messages in thread
From: Andrii Nakryiko @ 2021-01-11 21:03 UTC (permalink / raw)
  To: sedat.dilek, Jiri Olsa
  Cc: Yonghong Song, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend,
	KP Singh, Masahiro Yamada, bpf, Linux Kbuild mailing list,
	Tom Stellard

On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
> >
> >
> >
> > On 1/11/21 4:48 AM, Sedat Dilek wrote:
> > > Hi BPF maintainers and Mashiro,
> > >
> > > Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> > >
> > > My kernel-build fails like this:
> > >
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >   BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > The root cause is my selfmade LLVM toolchain has no BPF support.
> >
> > linux build should depend on LLVM toolchain unless you use LLVM to build
> > kernel.
> >
> > >
> > > $ which llc
> > > /home/dileks/src/llvm-toolchain/install/bin/llc
> > >
> > > $ llc --version
> > > LLVM (http://llvm.org/ ):
> > >   LLVM version 11.0.1
> > >   Optimized build.
> > >   Default target: x86_64-unknown-linux-gnu
> > >   Host CPU: sandybridge
> > >
> > >   Registered Targets:
> > >     x86    - 32-bit X86: Pentium-Pro and above
> > >     x86-64 - 64-bit X86: EM64T and AMD64
> > >
> > > Debian's llc-11 shows me BPF support is built-in.
> > >
> > > I see the breakag approx. 3 hours after the start of my kernel-build -
> > > in the stage "vmlinux".
> > > After 2 faulures in my build (2x 3 hours of build-time) I have still
> > > no finished Linux v5.11-rc3 kernel.
> > > This is a bit frustrating.
> >
> > You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> > Maybe a bug in resolve_btfids due to somehow different ELF format
> > resolve_btfids need to handle?
> >
>
> [ CC Tom ]
>
> OMG no.
>
> 3 hours up to running scripts/link-vmlinux.sh.
>
> In the meantime I have built a LLVM toolchain with BPF support.
>
> $ llc --version
> LLVM (http://llvm.org/):
>  LLVM version 11.0.1
>  Optimized build.
>  Default target: x86_64-unknown-linux-gnu
>  Host CPU: sandybridge
>
>  Registered Targets:
>    bpf    - BPF (host endian)
>    bpfeb  - BPF (big endian)
>    bpfel  - BPF (little endian)

As Yonghong mentioned, you don't need BPF target support in Clang to
build the kernel, so the issue is elsewhere. It's somewhere between
generated DWARF (we've seen multiple bugs in DWARF over time),
pahole's BTF output and resolve_btfids's handling of that BTF. I've
CC'ed Jiri, who can help with resolve_btfids.

Meanwhile, if you can provide SHA from which you built Clang, kernel
config you used, and probably exact invocation of the build you used,
it would help reproduce the issue.

>    x86    - 32-bit X86: Pentium-Pro and above
>    x86-64 - 64-bit X86: EM64T and AMD64
>
> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
> "I ran into this same bug trying to build the Fedora kernel. The
> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
> looks to me like a bug in pahole."
>
> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
> and use it.
>
> Building a new Linux-kernel...
>
> - Sedat -
>
> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553

There are no significant bug fixes between pahole 1.19 and master that
would solve this problem, so let's try to repro this.

>
>
>
> > >
> > > What about doing pre-checks - means before doing a single line of
> > > compilation - to check for:
> > > 1. Required binaries
> > > 2. Required support of whatever feature in compiler, linker, toolchain etc.
> > >
> > > Recently, I fell over depmod binary not found in my PATH - in one of
> > > the last steps (modfinal) of the kernel build.
> > >
> > > Any ideas to improve the situation?
> > > ( ...and please no RTFM, see links below. )
> > >
> > > Thanks.
> > >
> > > Regards,
> > > - Sedat -
> > >
> > >
> > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> > > [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> > > [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> > > [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> > >

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
       [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
@ 2021-01-11 22:31         ` Jiri Olsa
  2021-01-11 22:34           ` Tom Stellard
  2021-01-14 14:13           ` Sedat Dilek
  2021-01-12  0:57         ` Yonghong Song
  1 sibling, 2 replies; 19+ messages in thread
From: Jiri Olsa @ 2021-01-11 22:31 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:

SNIP

> > >
> > > Building a new Linux-kernel...
> > >
> > > - Sedat -
> > >
> > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> >
> > There are no significant bug fixes between pahole 1.19 and master that
> > would solve this problem, so let's try to repro this.
> >
> 
> You are right pahole fom latest Git does not solve the issue.
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>  BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument

hm, is there a .BTF section in vmlinux?

is this working over vmlinux:
 $ bpftool btf dump file ./vmlinux

do you have a verbose build output? I'd think pahole scream first..

jirka


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 22:31         ` Jiri Olsa
@ 2021-01-11 22:34           ` Tom Stellard
  2021-01-12 10:46             ` Jiri Olsa
  2021-01-14 14:13           ` Sedat Dilek
  1 sibling, 1 reply; 19+ messages in thread
From: Tom Stellard @ 2021-01-11 22:34 UTC (permalink / raw)
  To: Jiri Olsa, Sedat Dilek
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list

On 1/11/21 2:31 PM, Jiri Olsa wrote:
> On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> 
> SNIP
> 
>>>>
>>>> Building a new Linux-kernel...
>>>>
>>>> - Sedat -
>>>>
>>>> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
>>>> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
>>>> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
>>>
>>> There are no significant bug fixes between pahole 1.19 and master that
>>> would solve this problem, so let's try to repro this.
>>>
>>
>> You are right pahole fom latest Git does not solve the issue.
>>
>> + info BTFIDS vmlinux
>> + [  != silent_ ]
>> + printf   %-7s %s\n BTFIDS vmlinux
>>   BTFIDS  vmlinux
>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
>> FAILED: load BTF from vmlinux: Invalid argument
> 
> hm, is there a .BTF section in vmlinux?
> 
> is this working over vmlinux:
>   $ bpftool btf dump file ./vmlinux
> 
> do you have a verbose build output? I'd think pahole scream first..
> 

It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This 
is pretty easy for me to reproduce.  I have logs, what other information 
would be helpful?  How about a pahole backtrace?

-Tom

> jirka
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
       [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
  2021-01-11 22:31         ` Jiri Olsa
@ 2021-01-12  0:57         ` Yonghong Song
  2021-01-12  4:09           ` Sedat Dilek
  1 sibling, 1 reply; 19+ messages in thread
From: Yonghong Song @ 2021-01-12  0:57 UTC (permalink / raw)
  To: sedat.dilek, Andrii Nakryiko
  Cc: Jiri Olsa, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list, Tom Stellard



On 1/11/21 1:30 PM, Sedat Dilek wrote:
> On Mon, Jan 11, 2021 at 10:03 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>>
>>> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
>>>>
>>>>
>>>>
>>>> On 1/11/21 4:48 AM, Sedat Dilek wrote:
>>>>> Hi BPF maintainers and Mashiro,
>>>>>
>>>>> Debian started to use CONFIG_DEBUG_INFO_BTF=y.
>>>>>
>>>>> My kernel-build fails like this:
>>>>>
>>>>> + info BTFIDS vmlinux
>>>>> + [  != silent_ ]
>>>>> + printf   %-7s %s\n BTFIDS vmlinux
>>>>>    BTFIDS  vmlinux
>>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
>>>>> FAILED: load BTF from vmlinux: Invalid argument
>>>>>
>>>>> The root cause is my selfmade LLVM toolchain has no BPF support.
>>>>
>>>> linux build should depend on LLVM toolchain unless you use LLVM to build
>>>> kernel.
>>>>
>>>>>
>>>>> $ which llc
>>>>> /home/dileks/src/llvm-toolchain/install/bin/llc
>>>>>
>>>>> $ llc --version
>>>>> LLVM (http://llvm.org/  ):
>>>>>    LLVM version 11.0.1
>>>>>    Optimized build.
>>>>>    Default target: x86_64-unknown-linux-gnu
>>>>>    Host CPU: sandybridge
>>>>>
>>>>>    Registered Targets:
>>>>>      x86    - 32-bit X86: Pentium-Pro and above
>>>>>      x86-64 - 64-bit X86: EM64T and AMD64
>>>>>
>>>>> Debian's llc-11 shows me BPF support is built-in.
>>>>>
>>>>> I see the breakag approx. 3 hours after the start of my kernel-build -
>>>>> in the stage "vmlinux".
>>>>> After 2 faulures in my build (2x 3 hours of build-time) I have still
>>>>> no finished Linux v5.11-rc3 kernel.
>>>>> This is a bit frustrating.
>>>>
>>>> You mean "BTFIDS  vmlinux" takes more than 3 hours here?
>>>> Maybe a bug in resolve_btfids due to somehow different ELF format
>>>> resolve_btfids need to handle?
>>>>
>>>
>>> [ CC Tom ]
>>>
>>> OMG no.
>>>
>>> 3 hours up to running scripts/link-vmlinux.sh.
>>>
>>> In the meantime I have built a LLVM toolchain with BPF support.
>>>
>>> $ llc --version
>>> LLVM (http://llvm.org/ ):
>>>   LLVM version 11.0.1
>>>   Optimized build.
>>>   Default target: x86_64-unknown-linux-gnu
>>>   Host CPU: sandybridge
>>>
>>>   Registered Targets:
>>>     bpf    - BPF (host endian)
>>>     bpfeb  - BPF (big endian)
>>>     bpfel  - BPF (little endian)
>>
>> As Yonghong mentioned, you don't need BPF target support in Clang to
>> build the kernel, so the issue is elsewhere. It's somewhere between
>> generated DWARF (we've seen multiple bugs in DWARF over time),
>> pahole's BTF output and resolve_btfids's handling of that BTF. I've
>> CC'ed Jiri, who can help with resolve_btfids.
>>
>> Meanwhile, if you can provide SHA from which you built Clang, kernel
>> config you used, and probably exact invocation of the build you used,
>> it would help reproduce the issue.
>>
> 
> OK, I see I have here DWARF v5 support patchset applied and enabled.
> 
> Furthermore: I applied latest clang-cfi.
> 
> This is with LLVM v11.0.1 final aka 43ff75f2c3feef64f9d73328230d34dac8832a91.

Did you use llvm to compile kernel? If this is the case, latest pahole 
will segfault. I am using latest trunk llvm. It is possible that 
generated dwarf with llvm is different from generated dwarf with gcc
and pahole did not process it correctly. I did not get time to
debug this though.

> 
> My kernel-config is attached.
> 
> [1] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-1-ndesaulniers@google.com/
> [2] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-2-ndesaulniers@google.com/
> [3] https://github.com/samitolvanen/linux/commits/clang-cfi
> 
>>>     x86    - 32-bit X86: Pentium-43ff75f2c3feef64f9d73328230d34dac8832a91
> Pro and above
>>>     x86-64 - 64-bit X86: EM64T and AMD64
>>>
>>> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
>>> "I ran into this same bug trying to build the Fedora kBROKEN_5-11-rc3-CONFIG_DEBUG_INFO_BTF-y-FAILED-load-BTF-from-vmlinux.txt
> ernel. The
>>> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
>>> looks to me like a bug in pahole."
>>>
>>> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
>>> and use it.
>>>
>>> Building a new Linux-kernel...
>>>
>>> - Sedat -
>>>
>>> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
>>> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
>>> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
>>
>> There are no significant bug fixes between pahole 1.19 and master that
>> would solve this problem, so let's try to repro this.
>>
> 
> You are right pahole fom latest Git does not solve the issue.
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>   BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument
> 
> - Sedat -
> 
>>>
>>>
>>>
>>>>>
>>>>> What about doing pre-checks - means before doing a single line of
>>>>> compilation - to check for:
>>>>> 1. Required binaries
>>>>> 2. Required support of whatever feature in compiler, linker, toolchain etc.
>>>>>
>>>>> Recently, I fell over depmod binary not found in my PATH - in one of
>>>>> the last steps (modfinal) of the kernel build.
>>>>>
>>>>> Any ideas to improve the situation?
>>>>> ( ...and please no RTFM, see links below. )
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Regards,
>>>>> - Sedat -
>>>>>
>>>>>
>>>>> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
>>>>> [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
>>>>> [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
>>>>> [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
>>>>>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12  0:57         ` Yonghong Song
@ 2021-01-12  4:09           ` Sedat Dilek
  0 siblings, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12  4:09 UTC (permalink / raw)
  To: Yonghong Song
  Cc: Andrii Nakryiko, Jiri Olsa, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, John Fastabend,
	KP Singh, Masahiro Yamada, bpf, Linux Kbuild mailing list,
	Tom Stellard

On Tue, Jan 12, 2021 at 1:57 AM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 1/11/21 1:30 PM, Sedat Dilek wrote:
> > On Mon, Jan 11, 2021 at 10:03 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> >>
> >> On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >>>
> >>> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 1/11/21 4:48 AM, Sedat Dilek wrote:
> >>>>> Hi BPF maintainers and Mashiro,
> >>>>>
> >>>>> Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> >>>>>
> >>>>> My kernel-build fails like this:
> >>>>>
> >>>>> + info BTFIDS vmlinux
> >>>>> + [  != silent_ ]
> >>>>> + printf   %-7s %s\n BTFIDS vmlinux
> >>>>>    BTFIDS  vmlinux
> >>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> >>>>> FAILED: load BTF from vmlinux: Invalid argument
> >>>>>
> >>>>> The root cause is my selfmade LLVM toolchain has no BPF support.
> >>>>
> >>>> linux build should depend on LLVM toolchain unless you use LLVM to build
> >>>> kernel.
> >>>>
> >>>>>
> >>>>> $ which llc
> >>>>> /home/dileks/src/llvm-toolchain/install/bin/llc
> >>>>>
> >>>>> $ llc --version
> >>>>> LLVM (http://llvm.org/  ):
> >>>>>    LLVM version 11.0.1
> >>>>>    Optimized build.
> >>>>>    Default target: x86_64-unknown-linux-gnu
> >>>>>    Host CPU: sandybridge
> >>>>>
> >>>>>    Registered Targets:
> >>>>>      x86    - 32-bit X86: Pentium-Pro and above
> >>>>>      x86-64 - 64-bit X86: EM64T and AMD64
> >>>>>
> >>>>> Debian's llc-11 shows me BPF support is built-in.
> >>>>>
> >>>>> I see the breakag approx. 3 hours after the start of my kernel-build -
> >>>>> in the stage "vmlinux".
> >>>>> After 2 faulures in my build (2x 3 hours of build-time) I have still
> >>>>> no finished Linux v5.11-rc3 kernel.
> >>>>> This is a bit frustrating.
> >>>>
> >>>> You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> >>>> Maybe a bug in resolve_btfids due to somehow different ELF format
> >>>> resolve_btfids need to handle?
> >>>>
> >>>
> >>> [ CC Tom ]
> >>>
> >>> OMG no.
> >>>
> >>> 3 hours up to running scripts/link-vmlinux.sh.
> >>>
> >>> In the meantime I have built a LLVM toolchain with BPF support.
> >>>
> >>> $ llc --version
> >>> LLVM (http://llvm.org/ ):
> >>>   LLVM version 11.0.1
> >>>   Optimized build.
> >>>   Default target: x86_64-unknown-linux-gnu
> >>>   Host CPU: sandybridge
> >>>
> >>>   Registered Targets:
> >>>     bpf    - BPF (host endian)
> >>>     bpfeb  - BPF (big endian)
> >>>     bpfel  - BPF (little endian)
> >>
> >> As Yonghong mentioned, you don't need BPF target support in Clang to
> >> build the kernel, so the issue is elsewhere. It's somewhere between
> >> generated DWARF (we've seen multiple bugs in DWARF over time),
> >> pahole's BTF output and resolve_btfids's handling of that BTF. I've
> >> CC'ed Jiri, who can help with resolve_btfids.
> >>
> >> Meanwhile, if you can provide SHA from which you built Clang, kernel
> >> config you used, and probably exact invocation of the build you used,
> >> it would help reproduce the issue.
> >>
> >
> > OK, I see I have here DWARF v5 support patchset applied and enabled.
> >
> > Furthermore: I applied latest clang-cfi.
> >
> > This is with LLVM v11.0.1 final aka 43ff75f2c3feef64f9d73328230d34dac8832a91.
>
> Did you use llvm to compile kernel? If this is the case, latest pahole
> will segfault. I am using latest trunk llvm. It is possible that
> generated dwarf with llvm is different from generated dwarf with gcc
> and pahole did not process it correctly. I did not get time to
> debug this though.
>

Yes, I used LLVM/Clang to compile the kernel.

- Sedat -

> >
> > My kernel-config is attached.
> >
> > [1] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-1-ndesaulniers@google.com/
> > [2] https://patchwork.kernel.org/project/linux-kbuild/patch/20201204011129.2493105-2-ndesaulniers@google.com/
> > [3] https://github.com/samitolvanen/linux/commits/clang-cfi
> >
> >>>     x86    - 32-bit X86: Pentium-43ff75f2c3feef64f9d73328230d34dac8832a91
> > Pro and above
> >>>     x86-64 - 64-bit X86: EM64T and AMD64
> >>>
> >>> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
> >>> "I ran into this same bug trying to build the Fedora kBROKEN_5-11-rc3-CONFIG_DEBUG_INFO_BTF-y-FAILED-load-BTF-from-vmlinux.txt
> > ernel. The
> >>> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
> >>> looks to me like a bug in pahole."
> >>>
> >>> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
> >>> and use it.
> >>>
> >>> Building a new Linux-kernel...
> >>>
> >>> - Sedat -
> >>>
> >>> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> >>> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> >>> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> >>
> >> There are no significant bug fixes between pahole 1.19 and master that
> >> would solve this problem, so let's try to repro this.
> >>
> >
> > You are right pahole fom latest Git does not solve the issue.
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >   BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
> >
> > - Sedat -
> >
> >>>
> >>>
> >>>
> >>>>>
> >>>>> What about doing pre-checks - means before doing a single line of
> >>>>> compilation - to check for:
> >>>>> 1. Required binaries
> >>>>> 2. Required support of whatever feature in compiler, linker, toolchain etc.
> >>>>>
> >>>>> Recently, I fell over depmod binary not found in my PATH - in one of
> >>>>> the last steps (modfinal) of the kernel build.
> >>>>>
> >>>>> Any ideas to improve the situation?
> >>>>> ( ...and please no RTFM, see links below. )
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> Regards,
> >>>>> - Sedat -
> >>>>>
> >>>>>
> >>>>> [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> >>>>> [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> >>>>> [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> >>>>> [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> >>>>>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 22:34           ` Tom Stellard
@ 2021-01-12 10:46             ` Jiri Olsa
  2021-01-12 13:10               ` Jiri Olsa
  0 siblings, 1 reply; 19+ messages in thread
From: Jiri Olsa @ 2021-01-12 10:46 UTC (permalink / raw)
  To: Tom Stellard
  Cc: Sedat Dilek, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > 
> > SNIP
> > 
> > > > > 
> > > > > Building a new Linux-kernel...
> > > > > 
> > > > > - Sedat -
> > > > > 
> > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > 
> > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > would solve this problem, so let's try to repro this.
> > > > 
> > > 
> > > You are right pahole fom latest Git does not solve the issue.
> > > 
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >   BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> > 
> > hm, is there a .BTF section in vmlinux?
> > 
> > is this working over vmlinux:
> >   $ bpftool btf dump file ./vmlinux
> > 
> > do you have a verbose build output? I'd think pahole scream first..
> > 
> 
> It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> pretty easy for me to reproduce.  I have logs, what other information would
> be helpful?  How about a pahole backtrace?

that'd be great.. I'll try to reproduce, but with the latest clang
it will take me some time

jirka

> 
> -Tom
> 
> > jirka
> > 
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 10:46             ` Jiri Olsa
@ 2021-01-12 13:10               ` Jiri Olsa
  2021-01-12 16:14                 ` Sedat Dilek
  0 siblings, 1 reply; 19+ messages in thread
From: Jiri Olsa @ 2021-01-12 13:10 UTC (permalink / raw)
  To: Tom Stellard
  Cc: Sedat Dilek, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > 
> > > SNIP
> > > 
> > > > > > 
> > > > > > Building a new Linux-kernel...
> > > > > > 
> > > > > > - Sedat -
> > > > > > 
> > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > 
> > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > would solve this problem, so let's try to repro this.
> > > > > 
> > > > 
> > > > You are right pahole fom latest Git does not solve the issue.
> > > > 
> > > > + info BTFIDS vmlinux
> > > > + [  != silent_ ]
> > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > >   BTFIDS  vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > > 
> > > hm, is there a .BTF section in vmlinux?
> > > 
> > > is this working over vmlinux:
> > >   $ bpftool btf dump file ./vmlinux
> > > 
> > > do you have a verbose build output? I'd think pahole scream first..
> > > 
> > 
> > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > pretty easy for me to reproduce.  I have logs, what other information would
> > be helpful?  How about a pahole backtrace?
> 
> that'd be great.. I'll try to reproduce, but with the latest clang
> it will take me some time

reproduced, attached pahole patch fixes it for me,

looks like gcc never left function without name,
which does not seem to be the case for clang

I'll send full patch later today

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index c40f059580da..781fb35a2646 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -70,6 +70,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
 		return 0;
 	if (!elf_sym__value(sym))
 		return 0;
+	if (!elf_sym__name(sym, btfe->symtab))
+		return 0;
 
 	if (functions_cnt == functions_alloc) {
 		functions_alloc = max(1000, functions_alloc * 3 / 2);
@@ -620,9 +622,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
 		if (!has_arg_names(cu, &fn->proto))
 			continue;
 		if (functions_cnt) {
-			struct elf_function *func;
+			const char *name = function__name(fn, cu);
+			struct elf_function *func = NULL;
 
-			func = find_function(btfe, function__name(fn, cu));
+			if (name)
+				func = find_function(btfe, name);
 			if (!func || func->generated)
 				continue;
 			func->generated = true;


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 13:10               ` Jiri Olsa
@ 2021-01-12 16:14                 ` Sedat Dilek
  2021-01-12 16:21                   ` Jiri Olsa
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 16:14 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 2:10 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> > On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > >
> > > > SNIP
> > > >
> > > > > > >
> > > > > > > Building a new Linux-kernel...
> > > > > > >
> > > > > > > - Sedat -
> > > > > > >
> > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > >
> > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > would solve this problem, so let's try to repro this.
> > > > > >
> > > > >
> > > > > You are right pahole fom latest Git does not solve the issue.
> > > > >
> > > > > + info BTFIDS vmlinux
> > > > > + [  != silent_ ]
> > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > >   BTFIDS  vmlinux
> > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > >
> > > > hm, is there a .BTF section in vmlinux?
> > > >
> > > > is this working over vmlinux:
> > > >   $ bpftool btf dump file ./vmlinux
> > > >
> > > > do you have a verbose build output? I'd think pahole scream first..
> > > >
> > >
> > > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > > pretty easy for me to reproduce.  I have logs, what other information would
> > > be helpful?  How about a pahole backtrace?
> >
> > that'd be great.. I'll try to reproduce, but with the latest clang
> > it will take me some time
>
> reproduced, attached pahole patch fixes it for me,
>
> looks like gcc never left function without name,
> which does not seem to be the case for clang
>
> I'll send full patch later today
>

Thanks for the diff.

Unfortunately, it does not apply on latest pahole git.

$ git describe
v1.19-7-gb688e3597060

- Sedat -

> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index c40f059580da..781fb35a2646 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -70,6 +70,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
>                 return 0;
>         if (!elf_sym__value(sym))
>                 return 0;
> +       if (!elf_sym__name(sym, btfe->symtab))
> +               return 0;
>
>         if (functions_cnt == functions_alloc) {
>                 functions_alloc = max(1000, functions_alloc * 3 / 2);
> @@ -620,9 +622,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
>                 if (!has_arg_names(cu, &fn->proto))
>                         continue;
>                 if (functions_cnt) {
> -                       struct elf_function *func;
> +                       const char *name = function__name(fn, cu);
> +                       struct elf_function *func = NULL;
>
> -                       func = find_function(btfe, function__name(fn, cu));
> +                       if (name)
> +                               func = find_function(btfe, name);
>                         if (!func || func->generated)
>                                 continue;
>                         func->generated = true;
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 16:14                 ` Sedat Dilek
@ 2021-01-12 16:21                   ` Jiri Olsa
  2021-01-12 17:18                     ` Sedat Dilek
  2021-01-12 20:47                     ` Sedat Dilek
  0 siblings, 2 replies; 19+ messages in thread
From: Jiri Olsa @ 2021-01-12 16:21 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 05:14:42PM +0100, Sedat Dilek wrote:
> On Tue, Jan 12, 2021 at 2:10 PM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> > > On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > > > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > > >
> > > > > SNIP
> > > > >
> > > > > > > >
> > > > > > > > Building a new Linux-kernel...
> > > > > > > >
> > > > > > > > - Sedat -
> > > > > > > >
> > > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > > >
> > > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > > would solve this problem, so let's try to repro this.
> > > > > > >
> > > > > >
> > > > > > You are right pahole fom latest Git does not solve the issue.
> > > > > >
> > > > > > + info BTFIDS vmlinux
> > > > > > + [  != silent_ ]
> > > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > > >   BTFIDS  vmlinux
> > > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > >
> > > > > hm, is there a .BTF section in vmlinux?
> > > > >
> > > > > is this working over vmlinux:
> > > > >   $ bpftool btf dump file ./vmlinux
> > > > >
> > > > > do you have a verbose build output? I'd think pahole scream first..
> > > > >
> > > >
> > > > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > > > pretty easy for me to reproduce.  I have logs, what other information would
> > > > be helpful?  How about a pahole backtrace?
> > >
> > > that'd be great.. I'll try to reproduce, but with the latest clang
> > > it will take me some time
> >
> > reproduced, attached pahole patch fixes it for me,
> >
> > looks like gcc never left function without name,
> > which does not seem to be the case for clang
> >
> > I'll send full patch later today
> >
> 
> Thanks for the diff.
> 
> Unfortunately, it does not apply on latest pahole git.
> 
> $ git describe
> v1.19-7-gb688e3597060

sry wrong master.. how about this one

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index 333973054b61..17f7a14f2ef0 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -72,6 +72,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
 
 	if (elf_sym__type(sym) != STT_FUNC)
 		return 0;
+	if (!elf_sym__name(sym, btfe->symtab))
+		return 0;
 
 	if (functions_cnt == functions_alloc) {
 		functions_alloc = max(1000, functions_alloc * 3 / 2);
@@ -730,9 +732,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
 		if (!has_arg_names(cu, &fn->proto))
 			continue;
 		if (functions_cnt) {
-			struct elf_function *func;
+			const char *name = function__name(fn, cu);
+			struct elf_function *func = NULL;
 
-			func = find_function(btfe, function__name(fn, cu));
+			if (name)
+				func = find_function(btfe, name);
 			if (!func || func->generated)
 				continue;
 			func->generated = true;
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 16:21                   ` Jiri Olsa
@ 2021-01-12 17:18                     ` Sedat Dilek
  2021-01-12 20:47                     ` Sedat Dilek
  1 sibling, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 17:18 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 5:22 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Jan 12, 2021 at 05:14:42PM +0100, Sedat Dilek wrote:
> > On Tue, Jan 12, 2021 at 2:10 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Tue, Jan 12, 2021 at 11:46:22AM +0100, Jiri Olsa wrote:
> > > > On Mon, Jan 11, 2021 at 02:34:04PM -0800, Tom Stellard wrote:
> > > > > On 1/11/21 2:31 PM, Jiri Olsa wrote:
> > > > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > > > >
> > > > > > SNIP
> > > > > >
> > > > > > > > >
> > > > > > > > > Building a new Linux-kernel...
> > > > > > > > >
> > > > > > > > > - Sedat -
> > > > > > > > >
> > > > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > > > >
> > > > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > > > would solve this problem, so let's try to repro this.
> > > > > > > >
> > > > > > >
> > > > > > > You are right pahole fom latest Git does not solve the issue.
> > > > > > >
> > > > > > > + info BTFIDS vmlinux
> > > > > > > + [  != silent_ ]
> > > > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > > > >   BTFIDS  vmlinux
> > > > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > > >
> > > > > > hm, is there a .BTF section in vmlinux?
> > > > > >
> > > > > > is this working over vmlinux:
> > > > > >   $ bpftool btf dump file ./vmlinux
> > > > > >
> > > > > > do you have a verbose build output? I'd think pahole scream first..
> > > > > >
> > > > >
> > > > > It does.  For me, pahole segfaults at scripts/link-vmlinux.sh:131.  This is
> > > > > pretty easy for me to reproduce.  I have logs, what other information would
> > > > > be helpful?  How about a pahole backtrace?
> > > >
> > > > that'd be great.. I'll try to reproduce, but with the latest clang
> > > > it will take me some time
> > >
> > > reproduced, attached pahole patch fixes it for me,
> > >
> > > looks like gcc never left function without name,
> > > which does not seem to be the case for clang
> > >
> > > I'll send full patch later today
> > >
> >
> > Thanks for the diff.
> >
> > Unfortunately, it does not apply on latest pahole git.
> >
> > $ git describe
> > v1.19-7-gb688e3597060
>
> sry wrong master.. how about this one
>

I have applied the previous diff on top of tagsv1.19.

Thanks for the follow-up.

Please CC me on the patch you wanted to send later.

- Sedat -

> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index 333973054b61..17f7a14f2ef0 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -72,6 +72,8 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym)
>
>         if (elf_sym__type(sym) != STT_FUNC)
>                 return 0;
> +       if (!elf_sym__name(sym, btfe->symtab))
> +               return 0;
>
>         if (functions_cnt == functions_alloc) {
>                 functions_alloc = max(1000, functions_alloc * 3 / 2);
> @@ -730,9 +732,11 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
>                 if (!has_arg_names(cu, &fn->proto))
>                         continue;
>                 if (functions_cnt) {
> -                       struct elf_function *func;
> +                       const char *name = function__name(fn, cu);
> +                       struct elf_function *func = NULL;
>
> -                       func = find_function(btfe, function__name(fn, cu));
> +                       if (name)
> +                               func = find_function(btfe, name);
>                         if (!func || func->generated)
>                                 continue;
>                         func->generated = true;
> --
> 2.26.2
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 16:21                   ` Jiri Olsa
  2021-01-12 17:18                     ` Sedat Dilek
@ 2021-01-12 20:47                     ` Sedat Dilek
  2021-01-12 21:36                       ` Sedat Dilek
  1 sibling, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 20:47 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

Is it possible that someone tests with Nick Desaulniers's DWARF v5 patchset?

With b4 tool installed:

link="https://lore.kernel.org/r/CAKwvOdkZEiHK01OD420USb0j=F0LcrnRbauv9Yw26tu-GRbYkg@mail.gmail.com"
b4 -d am $link

...should give you a file which you can directly apply with git tool.

File: v3_20201203_ndesaulniers_kbuild_make_dwarf_version_a_choice.mbx

As my selfmade LLVM toolchain seems to be no good choice I jumped to gcc-10.

I see gazillions of DW_AT_ and DW_TAG_INVALID (0x48) warnings:

die__process_inline_expansion: DW_TAG_INVALID (0x48) @ <0x1c2a25e> not handled!

This might be due to DWARF version 5 patchset.
I did not check DWARF v2 (default) or DWARF v4 choices.

Beyond this, I noticed:

$ git grep LLVM_OBJCOPY scripts/ tools/
scripts/Makefile.modfinal:              LLVM_OBJCOPY=$(OBJCOPY)
$(PAHOLE) -J --btf_base vmlinux $@; \
scripts/link-vmlinux.sh:        LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
tools/testing/selftests/bpf/Makefile:LLVM_OBJCOPY       ?= llvm-objcopy

When only passing gcc-10 as compiler and ld.bfd as linker -
LLVM_OBJCOPY=objcopy is passed.
Means objcopy from GNU/binutils is used.
As far as I understand the code you want here llvm-objcopy
(scripts/Makefile.modfinal)?

I did a next tryout with gcc-10 and LLVM=1 means use all equivalent
LLVM tools to GNU/binutils.
Means use llvm-objcopy, llvm-nm, llvm-ar, llvm-objdump, etc.
Please see the link below under [1].

En plus, I tried pahole with Jirka "Convulted" Olsa (JCO) fix by passing...
   PAHOLE=/opt/pahole/bin/pahole
...to my make line.

Again I see that DW_TAG_INVALID (0x48) warnings.
Can someone enlighten me what that means?

Finally, with all that warnings observed I could "successfully"
generate Linux-kernel Debian packages.
I do NOT install software I do NOT trust - not booted - NoNoNo...

Thanks.

In the dark,
- Sedat -

[1] https://www.kernel.org/doc/html/latest/kbuild/llvm.html#llvm-utilities

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-12 20:47                     ` Sedat Dilek
@ 2021-01-12 21:36                       ` Sedat Dilek
  0 siblings, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-12 21:36 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Tom Stellard, Andrii Nakryiko, Jiri Olsa, Yonghong Song,
	Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, John Fastabend, KP Singh,
	Masahiro Yamada, bpf, Linux Kbuild mailing list

On Tue, Jan 12, 2021 at 9:47 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:

> En plus, I tried pahole with Jirka "Convulted" Olsa (JCO) fix by passing...
>    PAHOLE=/opt/pahole/bin/pahole
> ...to my make line.
>

Grrr, my selfmade pahole misses that patch.

How can I re-invoke make ... with new really fixed pahole version?

Fallen into a (pa)hole,
- Sedat -

P.S.: Download Jiri's patch

link="https://lore.kernel.org/bpf/20210112194724.GB1291051@krava/T/#t"
b4 -d am $link

- EOT -

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-11 22:31         ` Jiri Olsa
  2021-01-11 22:34           ` Tom Stellard
@ 2021-01-14 14:13           ` Sedat Dilek
  2021-01-14 14:24             ` Jiri Olsa
  1 sibling, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-14 14:13 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
>
> SNIP
>
> > > >
> > > > Building a new Linux-kernel...
> > > >
> > > > - Sedat -
> > > >
> > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > >
> > > There are no significant bug fixes between pahole 1.19 and master that
> > > would solve this problem, so let's try to repro this.
> > >
> >
> > You are right pahole fom latest Git does not solve the issue.
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >  BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
>
> hm, is there a .BTF section in vmlinux?
>
> is this working over vmlinux:
>  $ bpftool btf dump file ./vmlinux
>

I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.

The generated vmlinux file is cleaned on failure.

+ info BTFIDS vmlinux
+ [  != silent_ ]
+ printf   %-7s %s\n BTFIDS vmlinux
 BTFIDS  vmlinux
+ ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
FAILED: load BTF from vmlinux: Invalid argument
+ on_exit
+ [ 255 -ne 0 ]
+ cleanup
+ rm -f .btf.vmlinux.bin.o
+ rm -f .tmp_System.map
+ rm -f .tmp_initcalls.lds
+ rm -f .tmp_symversions.lds
+ rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
.tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
.tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
2.o
+ rm -f System.map
+ rm -f vmlinux
+ rm -f vmlinux.o

Dunno, how to suppress this.

- Sedat -

> do you have a verbose build output? I'd think pahole scream first..
>
> jirka
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-14 14:13           ` Sedat Dilek
@ 2021-01-14 14:24             ` Jiri Olsa
  2021-01-14 14:29               ` Sedat Dilek
  0 siblings, 1 reply; 19+ messages in thread
From: Jiri Olsa @ 2021-01-14 14:24 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Thu, Jan 14, 2021 at 03:13:18PM +0100, Sedat Dilek wrote:
> On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> >
> > SNIP
> >
> > > > >
> > > > > Building a new Linux-kernel...
> > > > >
> > > > > - Sedat -
> > > > >
> > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > >
> > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > would solve this problem, so let's try to repro this.
> > > >
> > >
> > > You are right pahole fom latest Git does not solve the issue.
> > >
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >  BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> >
> > hm, is there a .BTF section in vmlinux?
> >
> > is this working over vmlinux:
> >  $ bpftool btf dump file ./vmlinux
> >
> 
> I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.
> 
> The generated vmlinux file is cleaned on failure.
> 
> + info BTFIDS vmlinux
> + [  != silent_ ]
> + printf   %-7s %s\n BTFIDS vmlinux
>  BTFIDS  vmlinux
> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED: load BTF from vmlinux: Invalid argument

did pahole generated the .BTF section? earlier in the log

jirka


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
  2021-01-14 14:24             ` Jiri Olsa
@ 2021-01-14 14:29               ` Sedat Dilek
       [not found]                 ` <CA+icZUXp5gkxCr+mybQ8uO3xx+rVmmRSnN7Dfx0Xmr+VQWyZAA@mail.gmail.com>
  0 siblings, 1 reply; 19+ messages in thread
From: Sedat Dilek @ 2021-01-14 14:29 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

On Thu, Jan 14, 2021 at 3:24 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Thu, Jan 14, 2021 at 03:13:18PM +0100, Sedat Dilek wrote:
> > On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > >
> > > SNIP
> > >
> > > > > >
> > > > > > Building a new Linux-kernel...
> > > > > >
> > > > > > - Sedat -
> > > > > >
> > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > >
> > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > would solve this problem, so let's try to repro this.
> > > > >
> > > >
> > > > You are right pahole fom latest Git does not solve the issue.
> > > >
> > > > + info BTFIDS vmlinux
> > > > + [  != silent_ ]
> > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > >  BTFIDS  vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > hm, is there a .BTF section in vmlinux?
> > >
> > > is this working over vmlinux:
> > >  $ bpftool btf dump file ./vmlinux
> > >
> >
> > I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.
> >
> > The generated vmlinux file is cleaned on failure.
> >
> > + info BTFIDS vmlinux
> > + [  != silent_ ]
> > + printf   %-7s %s\n BTFIDS vmlinux
> >  BTFIDS  vmlinux
> > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED: load BTF from vmlinux: Invalid argument
>
> did pahole generated the .BTF section? earlier in the log
>
> jirka
>

Cannot say.

I did:

$ git diff
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 2d0b28758aa5..c6e679d08bbe 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -401,12 +401,6 @@ fi

vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}

-# fill in BTF IDs
-if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
-       info BTFIDS vmlinux
-       ${RESOLVE_BTFIDS} vmlinux
-fi
-
if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
       info SORTTAB vmlinux
       if ! sorttable vmlinux; then

- Sedat -

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
       [not found]                 ` <CA+icZUXp5gkxCr+mybQ8uO3xx+rVmmRSnN7Dfx0Xmr+VQWyZAA@mail.gmail.com>
@ 2021-01-14 15:13                   ` Sedat Dilek
  0 siblings, 0 replies; 19+ messages in thread
From: Sedat Dilek @ 2021-01-14 15:13 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Andrii Nakryiko, Jiri Olsa, Yonghong Song, Alexei Starovoitov,
	Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
	John Fastabend, KP Singh, Masahiro Yamada, bpf,
	Linux Kbuild mailing list, Tom Stellard

[-- Attachment #1: Type: text/plain, Size: 3509 bytes --]

On Thu, Jan 14, 2021 at 3:50 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Thu, Jan 14, 2021 at 3:29 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Thu, Jan 14, 2021 at 3:24 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Thu, Jan 14, 2021 at 03:13:18PM +0100, Sedat Dilek wrote:
> > > > On Mon, Jan 11, 2021 at 11:31 PM Jiri Olsa <jolsa@redhat.com> wrote:
> > > > >
> > > > > On Mon, Jan 11, 2021 at 10:30:22PM +0100, Sedat Dilek wrote:
> > > > >
> > > > > SNIP
> > > > >
> > > > > > > >
> > > > > > > > Building a new Linux-kernel...
> > > > > > > >
> > > > > > > > - Sedat -
> > > > > > > >
> > > > > > > > [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> > > > > > > > [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> > > > > > > > [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553
> > > > > > >
> > > > > > > There are no significant bug fixes between pahole 1.19 and master that
> > > > > > > would solve this problem, so let's try to repro this.
> > > > > > >
> > > > > >
> > > > > > You are right pahole fom latest Git does not solve the issue.
> > > > > >
> > > > > > + info BTFIDS vmlinux
> > > > > > + [  != silent_ ]
> > > > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > > > >  BTFIDS  vmlinux
> > > > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > >
> > > > > hm, is there a .BTF section in vmlinux?
> > > > >
> > > > > is this working over vmlinux:
> > > > >  $ bpftool btf dump file ./vmlinux
> > > > >
> > > >
> > > > I switched to LLVM v12 from <apt.llvm.org> and saw the same FAILED line.
> > > >
> > > > The generated vmlinux file is cleaned on failure.
> > > >
> > > > + info BTFIDS vmlinux
> > > > + [  != silent_ ]
> > > > + printf   %-7s %s\n BTFIDS vmlinux
> > > >  BTFIDS  vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > did pahole generated the .BTF section? earlier in the log
> > >
> > > jirka
> > >
> >
> > Cannot say.
> >
> > I did:
> >
> > $ git diff
> > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> > index 2d0b28758aa5..c6e679d08bbe 100755
> > --- a/scripts/link-vmlinux.sh
> > +++ b/scripts/link-vmlinux.sh
> > @@ -401,12 +401,6 @@ fi
> >
> > vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
> >
> > -# fill in BTF IDs
> > -if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
> > -       info BTFIDS vmlinux
> > -       ${RESOLVE_BTFIDS} vmlinux
> > -fi
> > -
> > if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
> >        info SORTTAB vmlinux
> >        if ! sorttable vmlinux; then
> >
>
> Before...
>
> Encountered error while encoding BTF.
> + llvm-objcopy --only-section=.BTF --set-section-flags
> .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
>
> ...I see lots of errors.
>
> I have cut the relevant stuff and placed it in the attachment.
>
> With freshly installed bpftool:
>
> $ /usr/sbin/bpftool btf dump file ./vmlinux
> Error: failed to load BTF from ./vmlinux: Invalid argument
>

This is with CONFIG_DEBUG_INFO_DWARF4=y.
( See attached patch with cover-letter - applied here on top of Linux
v5.11-rc3+).

Concerning this I am looking into [1]:

die__process_unit: DW_TAG_label (0xa) @ <0xa7> not handled!
die__process_unit: tag not supported 0xa (label)!

- Sedat -

[1] http://www.dwarfstd.org/doc/DWARF4.pdf > 3.5 LABEL ENTRIES

[-- Attachment #2: v4_20210112_ndesaulniers_kbuild_dwarf_v5_support.cover --]
[-- Type: application/octet-stream, Size: 3093 bytes --]

Sender: "ndesaulniers via sendgmr" <ndesaulniers@ndesaulniers1.mtv.corp.google.com>
Date: Tue, 12 Jan 2021 16:32:32 -0800
Message-Id: <20210113003235.716547-1-ndesaulniers@google.com>
Subject: [PATCH v4 0/3] Kbuild: DWARF v5 support
From: Nick Desaulniers <ndesaulniers@google.com>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <natechancellor@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, Sedat Dilek <sedat.dilek@gmail.com>, linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, linux-kbuild@vger.kernel.org, linux-arch@vger.kernel.org, Jakub Jelinek <jakub@redhat.com>, Fangrui Song <maskray@google.com>, Caroline Tice <cmtice@google.com>, Nick Clifton <nickc@redhat.com>, Nick Desaulniers <ndesaulniers@google.com>
List-ID: <linux-arch.vger.kernel.org>
X-Mailing-List: linux-arch@vger.kernel.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

DWARF v5 is the latest standard of the DWARF debug info format.

DWARF5 wins significantly in terms of size when mixed with compression
(CONFIG_DEBUG_INFO_COMPRESSED).

Link: http://www.dwarfstd.org/doc/DWARF5.pdf

Patch 1 is a cleanup from Masahiro and isn't DWARF v5 specific.
Patch 2 is a cleanup that lays the ground work and isn't DWARF
v5 specific.
Patch 3 implements Kconfig and Kbuild support for DWARFv5.

Changes from v3:

Changes as per Arvind:
* only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
* add -gdwarf-5 to Kconfig shell script.
* only run Kconfig shell script for Clang.

Apologies to Sedat and Nathan; I appreciate previous testing/review, but
I did no carry forward your Tested-by and Reviewed-by tags, as the
patches have changed too much IMO.

Changes from v2:
* Drop two of the earlier patches that have been accepted already.
* Add measurements with GCC 10.2 to commit message.
* Update help text as per Arvind with help from Caroline.
* Improve case/wording between DWARF Versions as per Masahiro.

Changes from the RFC:
* split patch in 3 patch series, include Fangrui's patch, too.
* prefer `DWARF vX` format, as per Fangrui.
* use spaces between assignment in Makefile as per Masahiro.
* simplify setting dwarf-version-y as per Masahiro.
* indent `prompt` in Kconfig change as per Masahiro.
* remove explicit default in Kconfig as per Masahiro.
* add comments to test_dwarf5_support.sh.
* change echo in test_dwarf5_support.sh as per Masahiro.
* remove -u from test_dwarf5_support.sh as per Masahiro.
* add a -gdwarf-5 cc-option check to Kconfig as per Jakub.

Masahiro Yamada (1):
  Remove $(cc-option,-gdwarf-4) dependency from CONFIG_DEBUG_INFO_DWARF4

Nick Desaulniers (2):
  Kbuild: make DWARF version a choice
  Kbuild: implement support for DWARF v5

 Makefile                          | 15 +++++++----
 include/asm-generic/vmlinux.lds.h |  6 ++++-
 lib/Kconfig.debug                 | 41 +++++++++++++++++++++++++------
 scripts/test_dwarf5_support.sh    |  9 +++++++
 4 files changed, 58 insertions(+), 13 deletions(-)
 create mode 100755 scripts/test_dwarf5_support.sh

-- 
2.30.0.284.gd98b1dd5eaa7-goog


[-- Attachment #3: v4_20210112_ndesaulniers_kbuild_dwarf_v5_support.mbx --]
[-- Type: application/octet-stream, Size: 12905 bytes --]

From MAILER-DAEMON Wed Jan 13 01:13:04 2021
Sender: "ndesaulniers via sendgmr" <ndesaulniers@ndesaulniers1.mtv.corp.google.com>
Date: Tue, 12 Jan 2021 16:32:33 -0800
In-Reply-To: <20210113003235.716547-1-ndesaulniers@google.com>
Message-Id: <20210113003235.716547-2-ndesaulniers@google.com>
References: <20210113003235.716547-1-ndesaulniers@google.com>
Subject: [PATCH v4 1/3] Remove $(cc-option,-gdwarf-4) dependency from CONFIG_DEBUG_INFO_DWARF4
From: Nick Desaulniers <ndesaulniers@google.com>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <natechancellor@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, Sedat Dilek <sedat.dilek@gmail.com>, linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, linux-kbuild@vger.kernel.org, linux-arch@vger.kernel.org, Jakub Jelinek <jakub@redhat.com>, Fangrui Song <maskray@google.com>, Caroline Tice <cmtice@google.com>, Nick Clifton <nickc@redhat.com>, Nick Desaulniers <ndesaulniers@google.com>
List-ID: <linux-arch.vger.kernel.org>
X-Mailing-List: linux-arch@vger.kernel.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

From: Masahiro Yamada <masahiroy@kernel.org>

The -gdwarf-4 flag is supported by GCC 4.5+, and also by Clang.

You can see it at https://godbolt.org/z/6ed1oW

  For gcc 4.5.3 pane,    line 37:    .value 0x4
  For clang 10.0.1 pane, line 117:   .short 4

Given Documentation/process/changes.rst stating GCC 4.9 is the minimal
version, this cc-option is unneeded.

Note
----

CONFIG_DEBUG_INFO_DWARF4 controls the DWARF version only for C files.

As you can see in the top Makefile, -gdwarf-4 is only passed to CFLAGS.

  ifdef CONFIG_DEBUG_INFO_DWARF4
  DEBUG_CFLAGS    += -gdwarf-4
  endif

This flag is used when compiling *.c files.

On the other hand, the assembler is always given -gdwarf-2.

  KBUILD_AFLAGS   += -Wa,-gdwarf-2

Hence, the debug info that comes from *.S files is always DWARF v2.
This is simply because GAS supported only -gdwarf-2 for a long time.

Recently, GAS gained the support for --dwarf-[3|4|5] options. [1]
And, also we have Clang integrated assembler. So, the debug info
for *.S files might be improved if we want.

In my understanding, the current code is intentional, not a bug.

[1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=31bf18645d98b4d3d7357353be840e320649a67d

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
---
 lib/Kconfig.debug | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 78361f0abe3a..dd7d8d35b2a5 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -258,7 +258,6 @@ config DEBUG_INFO_SPLIT
 
 config DEBUG_INFO_DWARF4
 	bool "Generate dwarf4 debuginfo"
-	depends on $(cc-option,-gdwarf-4)
 	help
 	  Generate dwarf4 debug info. This requires recent versions
 	  of gcc and gdb. It makes the debug information larger.
-- 
2.30.0.284.gd98b1dd5eaa7-goog

From MAILER-DAEMON Wed Jan 13 01:13:04 2021
Sender: "ndesaulniers via sendgmr" <ndesaulniers@ndesaulniers1.mtv.corp.google.com>
Date: Tue, 12 Jan 2021 16:32:34 -0800
In-Reply-To: <20210113003235.716547-1-ndesaulniers@google.com>
Message-Id: <20210113003235.716547-3-ndesaulniers@google.com>
References: <20210113003235.716547-1-ndesaulniers@google.com>
Subject: [PATCH v4 2/3] Kbuild: make DWARF version a choice
From: Nick Desaulniers <ndesaulniers@google.com>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <natechancellor@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, Sedat Dilek <sedat.dilek@gmail.com>, linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, linux-kbuild@vger.kernel.org, linux-arch@vger.kernel.org, Jakub Jelinek <jakub@redhat.com>, Fangrui Song <maskray@google.com>, Caroline Tice <cmtice@google.com>, Nick Clifton <nickc@redhat.com>, Nick Desaulniers <ndesaulniers@google.com>
List-ID: <linux-arch.vger.kernel.org>
X-Mailing-List: linux-arch@vger.kernel.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice. Adds an
explicit CONFIG_DEBUG_INFO_DWARF2, which is the default. Does so in a
way that's forward compatible with existing configs, and makes adding
future versions more straightforward.

Suggested-by: Fangrui Song <maskray@google.com>
Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 Makefile          | 14 +++++++++-----
 lib/Kconfig.debug | 21 ++++++++++++++++-----
 2 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/Makefile b/Makefile
index d49c3f39ceb4..656fff17b331 100644
--- a/Makefile
+++ b/Makefile
@@ -826,12 +826,16 @@ else
 DEBUG_CFLAGS	+= -g
 endif
 
-ifneq ($(LLVM_IAS),1)
-KBUILD_AFLAGS	+= -Wa,-gdwarf-2
+dwarf-version-$(CONFIG_DEBUG_INFO_DWARF2) := 2
+dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
+DEBUG_CFLAGS	+= -gdwarf-$(dwarf-version-y)
+ifneq ($(dwarf-version-y)$(LLVM_IAS),21)
+# Binutils 2.35+ required for -gdwarf-4+ support.
+dwarf-aflag	:= $(call as-option,-Wa$(comma)-gdwarf-$(dwarf-version-y))
+ifdef CONFIG_CC_IS_CLANG
+DEBUG_CFLAGS	+= $(dwarf-aflag)
 endif
-
-ifdef CONFIG_DEBUG_INFO_DWARF4
-DEBUG_CFLAGS	+= -gdwarf-4
+KBUILD_AFLAGS	+= $(dwarf-aflag)
 endif
 
 ifdef CONFIG_DEBUG_INFO_REDUCED
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index dd7d8d35b2a5..e80770fac4f0 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -256,13 +256,24 @@ config DEBUG_INFO_SPLIT
 	  to know about the .dwo files and include them.
 	  Incompatible with older versions of ccache.
 
+choice
+	prompt "DWARF version"
+	help
+	  Which version of DWARF debug info to emit.
+
+config DEBUG_INFO_DWARF2
+	bool "Generate DWARF Version 2 debuginfo"
+	help
+	  Generate DWARF v2 debug info.
+
 config DEBUG_INFO_DWARF4
-	bool "Generate dwarf4 debuginfo"
+	bool "Generate DWARF Version 4 debuginfo"
 	help
-	  Generate dwarf4 debug info. This requires recent versions
-	  of gcc and gdb. It makes the debug information larger.
-	  But it significantly improves the success of resolving
-	  variables in gdb on optimized code.
+	  Generate DWARF v4 debug info. This requires gcc 4.5+ and gdb 7.0+.
+	  It makes the debug information larger, but it significantly
+	  improves the success of resolving variables in gdb on optimized code.
+
+endchoice # "DWARF version"
 
 config DEBUG_INFO_BTF
 	bool "Generate BTF typeinfo"
-- 
2.30.0.284.gd98b1dd5eaa7-goog

From MAILER-DAEMON Wed Jan 13 01:13:04 2021
Sender: "ndesaulniers via sendgmr" <ndesaulniers@ndesaulniers1.mtv.corp.google.com>
Date: Tue, 12 Jan 2021 16:32:35 -0800
In-Reply-To: <20210113003235.716547-1-ndesaulniers@google.com>
Message-Id: <20210113003235.716547-4-ndesaulniers@google.com>
References: <20210113003235.716547-1-ndesaulniers@google.com>
Subject: [PATCH v4 3/3] Kbuild: implement support for DWARF v5
From: Nick Desaulniers <ndesaulniers@google.com>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <natechancellor@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, Sedat Dilek <sedat.dilek@gmail.com>, linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, linux-kbuild@vger.kernel.org, linux-arch@vger.kernel.org, Jakub Jelinek <jakub@redhat.com>, Fangrui Song <maskray@google.com>, Caroline Tice <cmtice@google.com>, Nick Clifton <nickc@redhat.com>, Nick Desaulniers <ndesaulniers@google.com>, Arvind Sankar <nivedita@alum.mit.edu>
List-ID: <linux-arch.vger.kernel.org>
X-Mailing-List: linux-arch@vger.kernel.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

DWARF v5 is the latest standard of the DWARF debug info format.

Feature detection of DWARF5 is onerous, especially given that we've
removed $(AS), so we must query $(CC) for DWARF5 assembler directive
support.  GNU `as` only recently gained support for specifying
-gdwarf-5.

The DWARF version of a binary can be validated with:
$ llvm-dwarfdump vmlinux | head -n 4 | grep version
or
$ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version

DWARF5 wins significantly in terms of size when mixed with compression
(CONFIG_DEBUG_INFO_COMPRESSED).

363M    vmlinux.clang12.dwarf5.compressed
434M    vmlinux.clang12.dwarf4.compressed
439M    vmlinux.clang12.dwarf2.compressed
457M    vmlinux.clang12.dwarf5
536M    vmlinux.clang12.dwarf4
548M    vmlinux.clang12.dwarf2

515M    vmlinux.gcc10.2.dwarf5.compressed
599M    vmlinux.gcc10.2.dwarf4.compressed
624M    vmlinux.gcc10.2.dwarf2.compressed
630M    vmlinux.gcc10.2.dwarf5
765M    vmlinux.gcc10.2.dwarf4
809M    vmlinux.gcc10.2.dwarf2

Though the quality of debug info is harder to quantify; size is not a
proxy for quality.

Jakub notes:
  All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
  option that enabled some small DWARF subset (initially only a few
  DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
  after DWARF 5 has been finalized) started emitting DWARF5 section
  headers and got most of the DWARF5 changes in...

Version check GCC so that we don't need to worry about the difference in
command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
validate the DWARF Version in the assembler feature detection script.

Link: http://www.dwarfstd.org/doc/DWARF5.pdf
Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
Suggested-by: Jakub Jelinek <jakub@redhat.com>
Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Suggested-by: Fangrui Song <maskray@google.com>
Suggested-by: Caroline Tice <cmtice@google.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 Makefile                          |  1 +
 include/asm-generic/vmlinux.lds.h |  6 +++++-
 lib/Kconfig.debug                 | 17 +++++++++++++++++
 scripts/test_dwarf5_support.sh    |  9 +++++++++
 4 files changed, 32 insertions(+), 1 deletion(-)
 create mode 100755 scripts/test_dwarf5_support.sh

diff --git a/Makefile b/Makefile
index 656fff17b331..1067cfd98249 100644
--- a/Makefile
+++ b/Makefile
@@ -828,6 +828,7 @@ endif
 
 dwarf-version-$(CONFIG_DEBUG_INFO_DWARF2) := 2
 dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
+dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
 DEBUG_CFLAGS	+= -gdwarf-$(dwarf-version-y)
 ifneq ($(dwarf-version-y)$(LLVM_IAS),21)
 # Binutils 2.35+ required for -gdwarf-4+ support.
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 49944f00d2b3..37dc4110875e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -843,7 +843,11 @@
 		.debug_types	0 : { *(.debug_types) }			\
 		/* DWARF 5 */						\
 		.debug_macro	0 : { *(.debug_macro) }			\
-		.debug_addr	0 : { *(.debug_addr) }
+		.debug_addr	0 : { *(.debug_addr) }			\
+		.debug_line_str	0 : { *(.debug_line_str) }		\
+		.debug_loclists	0 : { *(.debug_loclists) }		\
+		.debug_rnglists	0 : { *(.debug_rnglists) }		\
+		.debug_str_offsets	0 : { *(.debug_str_offsets) }
 
 /* Stabs debugging sections. */
 #define STABS_DEBUG							\
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index e80770fac4f0..60a4f5e27ada 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -273,6 +273,23 @@ config DEBUG_INFO_DWARF4
 	  It makes the debug information larger, but it significantly
 	  improves the success of resolving variables in gdb on optimized code.
 
+config DEBUG_INFO_DWARF5
+	bool "Generate DWARF Version 5 debuginfo"
+	depends on GCC_VERSION >= 50000 || CC_IS_CLANG
+	depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
+	help
+	  Generate DWARF v5 debug info. Requires binutils 2.35, gcc 5.0+ (gcc
+	  5.0+ accepts the -gdwarf-5 flag but only had partial support for some
+	  draft features until 7.0), and gdb 8.0+.
+
+	  Changes to the structure of debug info in Version 5 allow for around
+	  15-18% savings in resulting image and debug info section sizes as
+	  compared to DWARF Version 4. DWARF Version 5 standardizes previous
+	  extensions such as accelerators for symbol indexing and the format
+	  for fission (.dwo/.dwp) files. Users may not want to select this
+	  config if they rely on tooling that has not yet been updated to
+	  support DWARF Version 5.
+
 endchoice # "DWARF version"
 
 config DEBUG_INFO_BTF
diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
new file mode 100755
index 000000000000..142a1b5c7fa2
--- /dev/null
+++ b/scripts/test_dwarf5_support.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+# Test that assembler accepts -gdwarf-5 and .file 0 directives, which were bugs
+# in binutils < 2.35.
+# https://sourceware.org/bugzilla/show_bug.cgi?id=25612
+# https://sourceware.org/bugzilla/show_bug.cgi?id=25614
+set -e
+echo '.file 0 "filename"' | $* -gdwarf-5 -Wa,-gdwarf-5 -c -x assembler -o /dev/null -
-- 
2.30.0.284.gd98b1dd5eaa7-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2021-01-14 15:14 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-11 12:48 Check pahole availibity and BPF support of toolchain before starting a Linux kernel build Sedat Dilek
2021-01-11 16:05 ` Yonghong Song
2021-01-11 17:56   ` Sedat Dilek
2021-01-11 21:03     ` Andrii Nakryiko
     [not found]       ` <CA+icZUVrF_LCVhELbNLA7=FzEZK4=jk3QLD9XT2w5bQNo=nnOA@mail.gmail.com>
2021-01-11 22:31         ` Jiri Olsa
2021-01-11 22:34           ` Tom Stellard
2021-01-12 10:46             ` Jiri Olsa
2021-01-12 13:10               ` Jiri Olsa
2021-01-12 16:14                 ` Sedat Dilek
2021-01-12 16:21                   ` Jiri Olsa
2021-01-12 17:18                     ` Sedat Dilek
2021-01-12 20:47                     ` Sedat Dilek
2021-01-12 21:36                       ` Sedat Dilek
2021-01-14 14:13           ` Sedat Dilek
2021-01-14 14:24             ` Jiri Olsa
2021-01-14 14:29               ` Sedat Dilek
     [not found]                 ` <CA+icZUXp5gkxCr+mybQ8uO3xx+rVmmRSnN7Dfx0Xmr+VQWyZAA@mail.gmail.com>
2021-01-14 15:13                   ` Sedat Dilek
2021-01-12  0:57         ` Yonghong Song
2021-01-12  4:09           ` Sedat Dilek

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).