Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Nathan Chancellor <nathan@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>,
	John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@kernel.org>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Networking <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
	clang-built-linux <clang-built-linux@googlegroups.com>,
	Veronika Kabatova <vkabatov@redhat.com>,
	Jiri Olsa <jolsa@kernel.org>
Subject: Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM
Date: Mon, 8 Feb 2021 22:13:10 -0800	[thread overview]
Message-ID: <CAEf4BzYvri7wzRnGH_qQbavXOx5TfBA0qx4nYVnn=YNGv+vNVw@mail.gmail.com> (raw)
In-Reply-To: <CAEf4BzZV0-zx6YKUUKmecs=icnQNXJjTokdkSAoexm36za+wdA@mail.gmail.com>

On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > >
> > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > >                       LLVM=1 O=build/aarch64 defconfig
> > > >
> > > > $ scripts/config \
> > > >     --file build/aarch64/.config \
> > > >     -e BPF_SYSCALL \
> > > >     -e DEBUG_INFO_BTF \
> > > >     -e FTRACE \
> > > >     -e FUNCTION_TRACER
> > > >
> > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > ...
> > > > FAILED unresolved symbol vfs_truncate
> > > > ...
> > > >
> > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > although that seems obvious given that is what introduced
> > > > BTF_ID(func, vfs_truncate).
> > > >
> > > > I am using the latest pahole v1.20 and LLVM is at
> > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > have no idea what is going wrong here. Is this a known issue?
> > > >
> > >
> > > I'll skip the reproduction games this time and will just request the
> > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > and see what's going on. Thanks.
> > >
> >
> > Sure thing, let me know if this works. I uploaded in two places to make
> > it easier to grab:
> >
> > zstd compressed:
> > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> >
>
> Thanks. I clearly see at least one instance of seemingly well-formed
> vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> it. Which means it should have been generated in BTF, but it doesn't
> appear to be, so it does seem like a pahole bug. I (or someone else
> before me) will continue tomorrow.
>
> $ llvm-dwarfdump vmlinux
> ...
>
> 0x00052e6f:   DW_TAG_subprogram
>                 DW_AT_name      ("vfs_truncate")
>                 DW_AT_decl_file
> ("/home/nathan/cbl/src/linux/include/linux/fs.h")
>                 DW_AT_decl_line (2520)
>                 DW_AT_prototyped        (true)
>                 DW_AT_type      (0x000452cb "long int")
>                 DW_AT_declaration       (true)
>                 DW_AT_external  (true)
>
> 0x00052e7b:     DW_TAG_formal_parameter
>                   DW_AT_type    (0x00045fc6 "const path*")
>
> 0x00052e80:     DW_TAG_formal_parameter
>                   DW_AT_type    (0x00045213 "long long int")
>
> ...
>

... and here's the *only* other one (not marked as declaration, but I
thought we already handle that, Jiri?):

0x01d0da35:   DW_TAG_subprogram
                DW_AT_low_pc    (0xffff80001031f430)
                DW_AT_high_pc   (0xffff80001031f598)
                DW_AT_frame_base        (DW_OP_reg29)
                DW_AT_GNU_all_call_sites        (true)
                DW_AT_name      ("vfs_truncate")
                DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
                DW_AT_decl_line (69)
                DW_AT_prototyped        (true)
                DW_AT_type      (0x01cfdfe4 "long int")
                DW_AT_external  (true)

> $ llvm-readelf -s vmlinux | rg vfs_truncate
>  15013: ffff800011c22418     4 OBJECT  LOCAL  DEFAULT    24
> __BTF_ID__func__vfs_truncate__609
>  22531: ffff80001189fe0d     0 NOTYPE  LOCAL  DEFAULT    17
> __kstrtab_vfs_truncate
>  22532: ffff8000118a985b     0 NOTYPE  LOCAL  DEFAULT    17
> __kstrtabns_vfs_truncate
>  22534: ffff800011873b7c     0 NOTYPE  LOCAL  DEFAULT     8
> __ksymtab_vfs_truncate
> 176099: ffff80001031f430   360 FUNC    GLOBAL DEFAULT     2 vfs_truncate
>
> $ bpftool btf dump file vmlinux | rg vfs_truncate
> <nothing>
>
> > uncompressed:
> > https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv
> >
> > Cheers,
> > Nathan