Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build

From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: sedat.dilek@gmail.com, Jiri Olsa <jolsa@kernel.org>
Cc: Yonghong Song <yhs@fb.com>, Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@kernel.org>,
	Masahiro Yamada <masahiroy@kernel.org>, bpf <bpf@vger.kernel.org>,
	Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
	Tom Stellard <tstellar@redhat.com>
Subject: Re: Check pahole availibity and BPF support of toolchain before starting a Linux kernel build
Date: Mon, 11 Jan 2021 13:03:21 -0800	[thread overview]
Message-ID: <CAEf4BzaEA5aWeCCvHp7ASo9TdfotcBtqNGexirEynHDSo7ufgg@mail.gmail.com> (raw)
In-Reply-To: <CA+icZUVm6ZZveqVoS83SVXe1nqkqZVRjLO+SK1_nXHKkgh4yPQ@mail.gmail.com>

On Mon, Jan 11, 2021 at 9:56 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Mon, Jan 11, 2021 at 5:05 PM Yonghong Song <yhs@fb.com> wrote:
> >
> >
> >
> > On 1/11/21 4:48 AM, Sedat Dilek wrote:
> > > Hi BPF maintainers and Mashiro,
> > >
> > > Debian started to use CONFIG_DEBUG_INFO_BTF=y.
> > >
> > > My kernel-build fails like this:
> > >
> > > + info BTFIDS vmlinux
> > > + [  != silent_ ]
> > > + printf   %-7s %s\n BTFIDS vmlinux
> > >   BTFIDS  vmlinux
> > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED: load BTF from vmlinux: Invalid argument
> > >
> > > The root cause is my selfmade LLVM toolchain has no BPF support.
> >
> > linux build should depend on LLVM toolchain unless you use LLVM to build
> > kernel.
> >
> > >
> > > $ which llc
> > > /home/dileks/src/llvm-toolchain/install/bin/llc
> > >
> > > $ llc --version
> > > LLVM (http://llvm.org/ ):
> > >   LLVM version 11.0.1
> > >   Optimized build.
> > >   Default target: x86_64-unknown-linux-gnu
> > >   Host CPU: sandybridge
> > >
> > >   Registered Targets:
> > >     x86    - 32-bit X86: Pentium-Pro and above
> > >     x86-64 - 64-bit X86: EM64T and AMD64
> > >
> > > Debian's llc-11 shows me BPF support is built-in.
> > >
> > > I see the breakag approx. 3 hours after the start of my kernel-build -
> > > in the stage "vmlinux".
> > > After 2 faulures in my build (2x 3 hours of build-time) I have still
> > > no finished Linux v5.11-rc3 kernel.
> > > This is a bit frustrating.
> >
> > You mean "BTFIDS  vmlinux" takes more than 3 hours here?
> > Maybe a bug in resolve_btfids due to somehow different ELF format
> > resolve_btfids need to handle?
> >
>
> [ CC Tom ]
>
> OMG no.
>
> 3 hours up to running scripts/link-vmlinux.sh.
>
> In the meantime I have built a LLVM toolchain with BPF support.
>
> $ llc --version
> LLVM (http://llvm.org/):
>  LLVM version 11.0.1
>  Optimized build.
>  Default target: x86_64-unknown-linux-gnu
>  Host CPU: sandybridge
>
>  Registered Targets:
>    bpf    - BPF (host endian)
>    bpfeb  - BPF (big endian)
>    bpfel  - BPF (little endian)

As Yonghong mentioned, you don't need BPF target support in Clang to
build the kernel, so the issue is elsewhere. It's somewhere between
generated DWARF (we've seen multiple bugs in DWARF over time),
pahole's BTF output and resolve_btfids's handling of that BTF. I've
CC'ed Jiri, who can help with resolve_btfids.

Meanwhile, if you can provide SHA from which you built Clang, kernel
config you used, and probably exact invocation of the build you used,
it would help reproduce the issue.

>    x86    - 32-bit X86: Pentium-Pro and above
>    x86-64 - 64-bit X86: EM64T and AMD64
>
> Tom reported BTF issues with pahole v1.19 (see [2] and [3]):
> "I ran into this same bug trying to build the Fedora kernel. The
> problem is that pahole segfaults at: scripts/link-vmlinux.sh:131. This
> looks to me like a bug in pahole."
>
> pahole ToT (post v1.19) offers some BTF fixes - I have manually build
> and use it.
>
> Building a new Linux-kernel...
>
> - Sedat -
>
> [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
> [2] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758026878
> [3] https://github.com/ClangBuiltLinux/tc-build/issues/129#issuecomment-758056553

There are no significant bug fixes between pahole 1.19 and master that
would solve this problem, so let's try to repro this.

>
>
>
> > >
> > > What about doing pre-checks - means before doing a single line of
> > > compilation - to check for:
> > > 1. Required binaries
> > > 2. Required support of whatever feature in compiler, linker, toolchain etc.
> > >
> > > Recently, I fell over depmod binary not found in my PATH - in one of
> > > the last steps (modfinal) of the kernel build.
> > >
> > > Any ideas to improve the situation?
> > > ( ...and please no RTFM, see links below. )
> > >
> > > Thanks.
> > >
> > > Regards,
> > > - Sedat -
> > >
> > >
> > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh#n144
> > > [1] https://salsa.debian.org/kernel-team/linux/-/commit/929891281c61ce4403ddd869664c949692644a2f
> > > [2] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html?highlight=pahole#llvm
> > > [3] https://www.kernel.org/doc/html/latest/bpf/btf.html?highlight=pahole#btf-generation
> > >