FAILED unresolved symbol vfs_truncate on arm64 with LLVM

FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

Hi all,

Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ

$ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
                      LLVM=1 O=build/aarch64 defconfig

$ scripts/config \
    --file build/aarch64/.config \
    -e BPF_SYSCALL \
    -e DEBUG_INFO_BTF \
    -e FTRACE \
    -e FUNCTION_TRACER

$ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
                      LLVM=1 O=build/aarch64 olddefconfig all
...
FAILED unresolved symbol vfs_truncate
...

My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
although that seems obvious given that is what introduced
BTF_ID(func, vfs_truncate).

I am using the latest pahole v1.20 and LLVM is at
https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
although I can reproduce back to LLVM 10.0.1, which is the earliest
version that the kernel supports. I am very unfamiliar with BPF so I
have no idea what is going wrong here. Is this a known issue?

Cheers,
Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Hi all,
>
> Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
>
> $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
>                       LLVM=1 O=build/aarch64 defconfig
>
> $ scripts/config \
>     --file build/aarch64/.config \
>     -e BPF_SYSCALL \
>     -e DEBUG_INFO_BTF \
>     -e FTRACE \
>     -e FUNCTION_TRACER
>
> $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
>                       LLVM=1 O=build/aarch64 olddefconfig all
> ...
> FAILED unresolved symbol vfs_truncate
> ...
>
> My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> although that seems obvious given that is what introduced
> BTF_ID(func, vfs_truncate).
>
> I am using the latest pahole v1.20 and LLVM is at
> https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> although I can reproduce back to LLVM 10.0.1, which is the earliest
> version that the kernel supports. I am very unfamiliar with BPF so I
> have no idea what is going wrong here. Is this a known issue?
>

I'll skip the reproduction games this time and will just request the
vmlinux image. Please upload somewhere so that we can look at DWARF
and see what's going on. Thanks.

> Cheers,
> Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > Hi all,
> >
> > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> >
> > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> >                       LLVM=1 O=build/aarch64 defconfig
> >
> > $ scripts/config \
> >     --file build/aarch64/.config \
> >     -e BPF_SYSCALL \
> >     -e DEBUG_INFO_BTF \
> >     -e FTRACE \
> >     -e FUNCTION_TRACER
> >
> > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> >                       LLVM=1 O=build/aarch64 olddefconfig all
> > ...
> > FAILED unresolved symbol vfs_truncate
> > ...
> >
> > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > although that seems obvious given that is what introduced
> > BTF_ID(func, vfs_truncate).
> >
> > I am using the latest pahole v1.20 and LLVM is at
> > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > version that the kernel supports. I am very unfamiliar with BPF so I
> > have no idea what is going wrong here. Is this a known issue?
> >
> 
> I'll skip the reproduction games this time and will just request the
> vmlinux image. Please upload somewhere so that we can look at DWARF
> and see what's going on. Thanks.
> 

Sure thing, let me know if this works. I uploaded in two places to make
it easier to grab:

zstd compressed:
https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst

uncompressed:
https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv

Cheers,
Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > Hi all,
> > >
> > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > >
> > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > >                       LLVM=1 O=build/aarch64 defconfig
> > >
> > > $ scripts/config \
> > >     --file build/aarch64/.config \
> > >     -e BPF_SYSCALL \
> > >     -e DEBUG_INFO_BTF \
> > >     -e FTRACE \
> > >     -e FUNCTION_TRACER
> > >
> > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > ...
> > > FAILED unresolved symbol vfs_truncate
> > > ...
> > >
> > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > although that seems obvious given that is what introduced
> > > BTF_ID(func, vfs_truncate).
> > >
> > > I am using the latest pahole v1.20 and LLVM is at
> > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > have no idea what is going wrong here. Is this a known issue?
> > >
> >
> > I'll skip the reproduction games this time and will just request the
> > vmlinux image. Please upload somewhere so that we can look at DWARF
> > and see what's going on. Thanks.
> >
>
> Sure thing, let me know if this works. I uploaded in two places to make
> it easier to grab:
>
> zstd compressed:
> https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
>

Thanks. I clearly see at least one instance of seemingly well-formed
vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
it. Which means it should have been generated in BTF, but it doesn't
appear to be, so it does seem like a pahole bug. I (or someone else
before me) will continue tomorrow.

$ llvm-dwarfdump vmlinux
...

0x00052e6f:   DW_TAG_subprogram
                DW_AT_name      ("vfs_truncate")
                DW_AT_decl_file
("/home/nathan/cbl/src/linux/include/linux/fs.h")
                DW_AT_decl_line (2520)
                DW_AT_prototyped        (true)
                DW_AT_type      (0x000452cb "long int")
                DW_AT_declaration       (true)
                DW_AT_external  (true)

0x00052e7b:     DW_TAG_formal_parameter
                  DW_AT_type    (0x00045fc6 "const path*")

0x00052e80:     DW_TAG_formal_parameter
                  DW_AT_type    (0x00045213 "long long int")

...

$ llvm-readelf -s vmlinux | rg vfs_truncate
 15013: ffff800011c22418     4 OBJECT  LOCAL  DEFAULT    24
__BTF_ID__func__vfs_truncate__609
 22531: ffff80001189fe0d     0 NOTYPE  LOCAL  DEFAULT    17
__kstrtab_vfs_truncate
 22532: ffff8000118a985b     0 NOTYPE  LOCAL  DEFAULT    17
__kstrtabns_vfs_truncate
 22534: ffff800011873b7c     0 NOTYPE  LOCAL  DEFAULT     8
__ksymtab_vfs_truncate
176099: ffff80001031f430   360 FUNC    GLOBAL DEFAULT     2 vfs_truncate

$ bpftool btf dump file vmlinux | rg vfs_truncate
<nothing>

> uncompressed:
> https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv
>
> Cheers,
> Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > >
> > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > >                       LLVM=1 O=build/aarch64 defconfig
> > > >
> > > > $ scripts/config \
> > > >     --file build/aarch64/.config \
> > > >     -e BPF_SYSCALL \
> > > >     -e DEBUG_INFO_BTF \
> > > >     -e FTRACE \
> > > >     -e FUNCTION_TRACER
> > > >
> > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > ...
> > > > FAILED unresolved symbol vfs_truncate
> > > > ...
> > > >
> > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > although that seems obvious given that is what introduced
> > > > BTF_ID(func, vfs_truncate).
> > > >
> > > > I am using the latest pahole v1.20 and LLVM is at
> > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > have no idea what is going wrong here. Is this a known issue?
> > > >
> > >
> > > I'll skip the reproduction games this time and will just request the
> > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > and see what's going on. Thanks.
> > >
> >
> > Sure thing, let me know if this works. I uploaded in two places to make
> > it easier to grab:
> >
> > zstd compressed:
> > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> >
>
> Thanks. I clearly see at least one instance of seemingly well-formed
> vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> it. Which means it should have been generated in BTF, but it doesn't
> appear to be, so it does seem like a pahole bug. I (or someone else
> before me) will continue tomorrow.
>
> $ llvm-dwarfdump vmlinux
> ...
>
> 0x00052e6f:   DW_TAG_subprogram
>                 DW_AT_name      ("vfs_truncate")
>                 DW_AT_decl_file
> ("/home/nathan/cbl/src/linux/include/linux/fs.h")
>                 DW_AT_decl_line (2520)
>                 DW_AT_prototyped        (true)
>                 DW_AT_type      (0x000452cb "long int")
>                 DW_AT_declaration       (true)
>                 DW_AT_external  (true)
>
> 0x00052e7b:     DW_TAG_formal_parameter
>                   DW_AT_type    (0x00045fc6 "const path*")
>
> 0x00052e80:     DW_TAG_formal_parameter
>                   DW_AT_type    (0x00045213 "long long int")
>
> ...
>

... and here's the *only* other one (not marked as declaration, but I
thought we already handle that, Jiri?):

0x01d0da35:   DW_TAG_subprogram
                DW_AT_low_pc    (0xffff80001031f430)
                DW_AT_high_pc   (0xffff80001031f598)
                DW_AT_frame_base        (DW_OP_reg29)
                DW_AT_GNU_all_call_sites        (true)
                DW_AT_name      ("vfs_truncate")
                DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
                DW_AT_decl_line (69)
                DW_AT_prototyped        (true)
                DW_AT_type      (0x01cfdfe4 "long int")
                DW_AT_external  (true)


> $ llvm-readelf -s vmlinux | rg vfs_truncate
>  15013: ffff800011c22418     4 OBJECT  LOCAL  DEFAULT    24
> __BTF_ID__func__vfs_truncate__609
>  22531: ffff80001189fe0d     0 NOTYPE  LOCAL  DEFAULT    17
> __kstrtab_vfs_truncate
>  22532: ffff8000118a985b     0 NOTYPE  LOCAL  DEFAULT    17
> __kstrtabns_vfs_truncate
>  22534: ffff800011873b7c     0 NOTYPE  LOCAL  DEFAULT     8
> __ksymtab_vfs_truncate
> 176099: ffff80001031f430   360 FUNC    GLOBAL DEFAULT     2 vfs_truncate
>
> $ bpftool btf dump file vmlinux | rg vfs_truncate
> <nothing>
>
> > uncompressed:
> > https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv
> >
> > Cheers,
> > Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > >
> > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > >
> > > > > $ scripts/config \
> > > > >     --file build/aarch64/.config \
> > > > >     -e BPF_SYSCALL \
> > > > >     -e DEBUG_INFO_BTF \
> > > > >     -e FTRACE \
> > > > >     -e FUNCTION_TRACER
> > > > >
> > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > ...
> > > > > FAILED unresolved symbol vfs_truncate
> > > > > ...
> > > > >
> > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > although that seems obvious given that is what introduced
> > > > > BTF_ID(func, vfs_truncate).
> > > > >
> > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > have no idea what is going wrong here. Is this a known issue?
> > > > >
> > > >
> > > > I'll skip the reproduction games this time and will just request the
> > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > and see what's going on. Thanks.
> > > >
> > >
> > > Sure thing, let me know if this works. I uploaded in two places to make
> > > it easier to grab:
> > >
> > > zstd compressed:
> > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > >
> >
> > Thanks. I clearly see at least one instance of seemingly well-formed
> > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > it. Which means it should have been generated in BTF, but it doesn't
> > appear to be, so it does seem like a pahole bug. I (or someone else
> > before me) will continue tomorrow.
> >
> > $ llvm-dwarfdump vmlinux
> > ...
> >
> > 0x00052e6f:   DW_TAG_subprogram
> >                 DW_AT_name      ("vfs_truncate")
> >                 DW_AT_decl_file
> > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> >                 DW_AT_decl_line (2520)
> >                 DW_AT_prototyped        (true)
> >                 DW_AT_type      (0x000452cb "long int")
> >                 DW_AT_declaration       (true)
> >                 DW_AT_external  (true)
> >
> > 0x00052e7b:     DW_TAG_formal_parameter
> >                   DW_AT_type    (0x00045fc6 "const path*")
> >
> > 0x00052e80:     DW_TAG_formal_parameter
> >                   DW_AT_type    (0x00045213 "long long int")
> >
> > ...
> >
>
> ... and here's the *only* other one (not marked as declaration, but I
> thought we already handle that, Jiri?):
>
> 0x01d0da35:   DW_TAG_subprogram
>                 DW_AT_low_pc    (0xffff80001031f430)
>                 DW_AT_high_pc   (0xffff80001031f598)
>                 DW_AT_frame_base        (DW_OP_reg29)
>                 DW_AT_GNU_all_call_sites        (true)
>                 DW_AT_name      ("vfs_truncate")
>                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
>                 DW_AT_decl_line (69)
>                 DW_AT_prototyped        (true)
>                 DW_AT_type      (0x01cfdfe4 "long int")
>                 DW_AT_external  (true)
>

Ok, the problem appears to be not in DWARF, but in mcount_loc data.
vfs_truncate's address is not recorded as ftrace-attachable, and thus
pahole ignores it. I don't know why this happens and it's quite
strange, given vfs_truncate is just a normal global function.

I'd like to understand this issue before we try to fix it, but there
is at least one improvement we can make: pahole should check ftrace
addresses only for static functions, not the global ones (global ones
should be always attachable, unless they are special, e.g., notrace
and stuff). We can easily check that by looking at the corresponding
symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
for that particular kernel. For that we'll need Nathan's cooperation,
unless someone else can build an arm64 kernel with the same problem
and check.

>
> > $ llvm-readelf -s vmlinux | rg vfs_truncate
> >  15013: ffff800011c22418     4 OBJECT  LOCAL  DEFAULT    24
> > __BTF_ID__func__vfs_truncate__609
> >  22531: ffff80001189fe0d     0 NOTYPE  LOCAL  DEFAULT    17
> > __kstrtab_vfs_truncate
> >  22532: ffff8000118a985b     0 NOTYPE  LOCAL  DEFAULT    17
> > __kstrtabns_vfs_truncate
> >  22534: ffff800011873b7c     0 NOTYPE  LOCAL  DEFAULT     8
> > __ksymtab_vfs_truncate
> > 176099: ffff80001031f430   360 FUNC    GLOBAL DEFAULT     2 vfs_truncate
> >
> > $ bpftool btf dump file vmlinux | rg vfs_truncate
> > <nothing>
> >
> > > uncompressed:
> > > https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv
> > >
> > > Cheers,
> > > Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Mon, Feb 08, 2021 at 10:56:36PM -0800, Andrii Nakryiko wrote:
> On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > >
> > > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > > >
> > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > > >
> > > > > > $ scripts/config \
> > > > > >     --file build/aarch64/.config \
> > > > > >     -e BPF_SYSCALL \
> > > > > >     -e DEBUG_INFO_BTF \
> > > > > >     -e FTRACE \
> > > > > >     -e FUNCTION_TRACER
> > > > > >
> > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > > ...
> > > > > > FAILED unresolved symbol vfs_truncate
> > > > > > ...
> > > > > >
> > > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > > although that seems obvious given that is what introduced
> > > > > > BTF_ID(func, vfs_truncate).
> > > > > >
> > > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > > have no idea what is going wrong here. Is this a known issue?
> > > > > >
> > > > >
> > > > > I'll skip the reproduction games this time and will just request the
> > > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > > and see what's going on. Thanks.
> > > > >
> > > >
> > > > Sure thing, let me know if this works. I uploaded in two places to make
> > > > it easier to grab:
> > > >
> > > > zstd compressed:
> > > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > > >
> > >
> > > Thanks. I clearly see at least one instance of seemingly well-formed
> > > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > > it. Which means it should have been generated in BTF, but it doesn't
> > > appear to be, so it does seem like a pahole bug. I (or someone else
> > > before me) will continue tomorrow.
> > >
> > > $ llvm-dwarfdump vmlinux
> > > ...
> > >
> > > 0x00052e6f:   DW_TAG_subprogram
> > >                 DW_AT_name      ("vfs_truncate")
> > >                 DW_AT_decl_file
> > > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> > >                 DW_AT_decl_line (2520)
> > >                 DW_AT_prototyped        (true)
> > >                 DW_AT_type      (0x000452cb "long int")
> > >                 DW_AT_declaration       (true)
> > >                 DW_AT_external  (true)
> > >
> > > 0x00052e7b:     DW_TAG_formal_parameter
> > >                   DW_AT_type    (0x00045fc6 "const path*")
> > >
> > > 0x00052e80:     DW_TAG_formal_parameter
> > >                   DW_AT_type    (0x00045213 "long long int")
> > >
> > > ...
> > >
> >
> > ... and here's the *only* other one (not marked as declaration, but I
> > thought we already handle that, Jiri?):
> >
> > 0x01d0da35:   DW_TAG_subprogram
> >                 DW_AT_low_pc    (0xffff80001031f430)
> >                 DW_AT_high_pc   (0xffff80001031f598)
> >                 DW_AT_frame_base        (DW_OP_reg29)
> >                 DW_AT_GNU_all_call_sites        (true)
> >                 DW_AT_name      ("vfs_truncate")
> >                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
> >                 DW_AT_decl_line (69)
> >                 DW_AT_prototyped        (true)
> >                 DW_AT_type      (0x01cfdfe4 "long int")
> >                 DW_AT_external  (true)
> >
> 
> Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> vfs_truncate's address is not recorded as ftrace-attachable, and thus
> pahole ignores it. I don't know why this happens and it's quite
> strange, given vfs_truncate is just a normal global function.
> 
> I'd like to understand this issue before we try to fix it, but there
> is at least one improvement we can make: pahole should check ftrace
> addresses only for static functions, not the global ones (global ones
> should be always attachable, unless they are special, e.g., notrace
> and stuff). We can easily check that by looking at the corresponding
> symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> for that particular kernel. For that we'll need Nathan's cooperation,
> unless someone else can build an arm64 kernel with the same problem
> and check.

Sure, just let me know what I need to do and I can do it!

Cheers,
Nathan

> >
> > > $ llvm-readelf -s vmlinux | rg vfs_truncate
> > >  15013: ffff800011c22418     4 OBJECT  LOCAL  DEFAULT    24
> > > __BTF_ID__func__vfs_truncate__609
> > >  22531: ffff80001189fe0d     0 NOTYPE  LOCAL  DEFAULT    17
> > > __kstrtab_vfs_truncate
> > >  22532: ffff8000118a985b     0 NOTYPE  LOCAL  DEFAULT    17
> > > __kstrtabns_vfs_truncate
> > >  22534: ffff800011873b7c     0 NOTYPE  LOCAL  DEFAULT     8
> > > __ksymtab_vfs_truncate
> > > 176099: ffff80001031f430   360 FUNC    GLOBAL DEFAULT     2 vfs_truncate
> > >
> > > $ bpftool btf dump file vmlinux | rg vfs_truncate
> > > <nothing>
> > >
> > > > uncompressed:
> > > > https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv
> > > >
> > > > Cheers,
> > > > Nathan
> 
> -- 
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAEf4Bzax90hn_5axpnCpW%2BE6gVc1mtUgCXWqmxV0tJ4Ud7bsaA%40mail.gmail.com.

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 12:49:04AM -0700, Nathan Chancellor wrote:
> On Mon, Feb 08, 2021 at 10:56:36PM -0800, Andrii Nakryiko wrote:
> > On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > >
> > > > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > > > >
> > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > > > >
> > > > > > > $ scripts/config \
> > > > > > >     --file build/aarch64/.config \
> > > > > > >     -e BPF_SYSCALL \
> > > > > > >     -e DEBUG_INFO_BTF \
> > > > > > >     -e FTRACE \
> > > > > > >     -e FUNCTION_TRACER
> > > > > > >
> > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > > > ...
> > > > > > > FAILED unresolved symbol vfs_truncate
> > > > > > > ...
> > > > > > >
> > > > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > > > although that seems obvious given that is what introduced
> > > > > > > BTF_ID(func, vfs_truncate).
> > > > > > >
> > > > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > > > have no idea what is going wrong here. Is this a known issue?
> > > > > > >
> > > > > >
> > > > > > I'll skip the reproduction games this time and will just request the
> > > > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > > > and see what's going on. Thanks.
> > > > > >
> > > > >
> > > > > Sure thing, let me know if this works. I uploaded in two places to make
> > > > > it easier to grab:
> > > > >
> > > > > zstd compressed:
> > > > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > > > >
> > > >
> > > > Thanks. I clearly see at least one instance of seemingly well-formed
> > > > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > > > it. Which means it should have been generated in BTF, but it doesn't
> > > > appear to be, so it does seem like a pahole bug. I (or someone else
> > > > before me) will continue tomorrow.
> > > >
> > > > $ llvm-dwarfdump vmlinux
> > > > ...
> > > >
> > > > 0x00052e6f:   DW_TAG_subprogram
> > > >                 DW_AT_name      ("vfs_truncate")
> > > >                 DW_AT_decl_file
> > > > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> > > >                 DW_AT_decl_line (2520)
> > > >                 DW_AT_prototyped        (true)
> > > >                 DW_AT_type      (0x000452cb "long int")
> > > >                 DW_AT_declaration       (true)
> > > >                 DW_AT_external  (true)
> > > >
> > > > 0x00052e7b:     DW_TAG_formal_parameter
> > > >                   DW_AT_type    (0x00045fc6 "const path*")
> > > >
> > > > 0x00052e80:     DW_TAG_formal_parameter
> > > >                   DW_AT_type    (0x00045213 "long long int")
> > > >
> > > > ...
> > > >
> > >
> > > ... and here's the *only* other one (not marked as declaration, but I
> > > thought we already handle that, Jiri?):
> > >
> > > 0x01d0da35:   DW_TAG_subprogram
> > >                 DW_AT_low_pc    (0xffff80001031f430)
> > >                 DW_AT_high_pc   (0xffff80001031f598)
> > >                 DW_AT_frame_base        (DW_OP_reg29)
> > >                 DW_AT_GNU_all_call_sites        (true)
> > >                 DW_AT_name      ("vfs_truncate")
> > >                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
> > >                 DW_AT_decl_line (69)
> > >                 DW_AT_prototyped        (true)
> > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > >                 DW_AT_external  (true)
> > >
> > 
> > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > pahole ignores it. I don't know why this happens and it's quite
> > strange, given vfs_truncate is just a normal global function.

right, I can't see it in mcount adresses.. but it begins with instructions
that appears to be nops, which would suggest it's traceable

	ffff80001031f430 <vfs_truncate>:
	ffff80001031f430: 5f 24 03 d5   hint    #34
	ffff80001031f434: 1f 20 03 d5   nop
	ffff80001031f438: 1f 20 03 d5   nop
	ffff80001031f43c: 3f 23 03 d5   hint    #25

> > 
> > I'd like to understand this issue before we try to fix it, but there
> > is at least one improvement we can make: pahole should check ftrace
> > addresses only for static functions, not the global ones (global ones
> > should be always attachable, unless they are special, e.g., notrace
> > and stuff). We can easily check that by looking at the corresponding
> > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > for that particular kernel. For that we'll need Nathan's cooperation,
> > unless someone else can build an arm64 kernel with the same problem
> > and check.

I'm building on arm now, let's see

> 
> Sure, just let me know what I need to do and I can do it!


could you please verify that you don't see vfs_truncate in
available_filter_functions?

 # cat /sys/kernel/debug/tracing/available_filter_functions | grep vfs_truncate

thanks,
jirka

> 
> Cheers,
> Nathan
> 
> > >
> > > > $ llvm-readelf -s vmlinux | rg vfs_truncate
> > > >  15013: ffff800011c22418     4 OBJECT  LOCAL  DEFAULT    24
> > > > __BTF_ID__func__vfs_truncate__609
> > > >  22531: ffff80001189fe0d     0 NOTYPE  LOCAL  DEFAULT    17
> > > > __kstrtab_vfs_truncate
> > > >  22532: ffff8000118a985b     0 NOTYPE  LOCAL  DEFAULT    17
> > > > __kstrtabns_vfs_truncate
> > > >  22534: ffff800011873b7c     0 NOTYPE  LOCAL  DEFAULT     8
> > > > __ksymtab_vfs_truncate
> > > > 176099: ffff80001031f430   360 FUNC    GLOBAL DEFAULT     2 vfs_truncate
> > > >
> > > > $ bpftool btf dump file vmlinux | rg vfs_truncate
> > > > <nothing>
> > > >
> > > > > uncompressed:
> > > > > https://1drv.ms/u/s!AsQNYeB-IEbqjQiUOspbEdXx49o7?e=ipA9Hv
> > > > >
> > > > > Cheers,
> > > > > Nathan
> > 
> > -- 
> > You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAEf4Bzax90hn_5axpnCpW%2BE6gVc1mtUgCXWqmxV0tJ4Ud7bsaA%40mail.gmail.com.
> 

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 01:36:41PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 12:49:04AM -0700, Nathan Chancellor wrote:
> > On Mon, Feb 08, 2021 at 10:56:36PM -0800, Andrii Nakryiko wrote:
> > > On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> > > > <andrii.nakryiko@gmail.com> wrote:
> > > > >
> > > > > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > >
> > > > > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > > >
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > > > > >
> > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > > > > >
> > > > > > > > $ scripts/config \
> > > > > > > >     --file build/aarch64/.config \
> > > > > > > >     -e BPF_SYSCALL \
> > > > > > > >     -e DEBUG_INFO_BTF \
> > > > > > > >     -e FTRACE \
> > > > > > > >     -e FUNCTION_TRACER
> > > > > > > >
> > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > > > > ...
> > > > > > > > FAILED unresolved symbol vfs_truncate
> > > > > > > > ...
> > > > > > > >
> > > > > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > > > > although that seems obvious given that is what introduced
> > > > > > > > BTF_ID(func, vfs_truncate).
> > > > > > > >
> > > > > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > > > > have no idea what is going wrong here. Is this a known issue?
> > > > > > > >
> > > > > > >
> > > > > > > I'll skip the reproduction games this time and will just request the
> > > > > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > > > > and see what's going on. Thanks.
> > > > > > >
> > > > > >
> > > > > > Sure thing, let me know if this works. I uploaded in two places to make
> > > > > > it easier to grab:
> > > > > >
> > > > > > zstd compressed:
> > > > > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > > > > >
> > > > >
> > > > > Thanks. I clearly see at least one instance of seemingly well-formed
> > > > > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > > > > it. Which means it should have been generated in BTF, but it doesn't
> > > > > appear to be, so it does seem like a pahole bug. I (or someone else
> > > > > before me) will continue tomorrow.
> > > > >
> > > > > $ llvm-dwarfdump vmlinux
> > > > > ...
> > > > >
> > > > > 0x00052e6f:   DW_TAG_subprogram
> > > > >                 DW_AT_name      ("vfs_truncate")
> > > > >                 DW_AT_decl_file
> > > > > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> > > > >                 DW_AT_decl_line (2520)
> > > > >                 DW_AT_prototyped        (true)
> > > > >                 DW_AT_type      (0x000452cb "long int")
> > > > >                 DW_AT_declaration       (true)
> > > > >                 DW_AT_external  (true)
> > > > >
> > > > > 0x00052e7b:     DW_TAG_formal_parameter
> > > > >                   DW_AT_type    (0x00045fc6 "const path*")
> > > > >
> > > > > 0x00052e80:     DW_TAG_formal_parameter
> > > > >                   DW_AT_type    (0x00045213 "long long int")
> > > > >
> > > > > ...
> > > > >
> > > >
> > > > ... and here's the *only* other one (not marked as declaration, but I
> > > > thought we already handle that, Jiri?):
> > > >
> > > > 0x01d0da35:   DW_TAG_subprogram
> > > >                 DW_AT_low_pc    (0xffff80001031f430)
> > > >                 DW_AT_high_pc   (0xffff80001031f598)
> > > >                 DW_AT_frame_base        (DW_OP_reg29)
> > > >                 DW_AT_GNU_all_call_sites        (true)
> > > >                 DW_AT_name      ("vfs_truncate")
> > > >                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
> > > >                 DW_AT_decl_line (69)
> > > >                 DW_AT_prototyped        (true)
> > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > >                 DW_AT_external  (true)
> > > >
> > > 
> > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > pahole ignores it. I don't know why this happens and it's quite
> > > strange, given vfs_truncate is just a normal global function.
> 
> right, I can't see it in mcount adresses.. but it begins with instructions
> that appears to be nops, which would suggest it's traceable
> 
> 	ffff80001031f430 <vfs_truncate>:
> 	ffff80001031f430: 5f 24 03 d5   hint    #34
> 	ffff80001031f434: 1f 20 03 d5   nop
> 	ffff80001031f438: 1f 20 03 d5   nop
> 	ffff80001031f43c: 3f 23 03 d5   hint    #25
> 
> > > 
> > > I'd like to understand this issue before we try to fix it, but there
> > > is at least one improvement we can make: pahole should check ftrace
> > > addresses only for static functions, not the global ones (global ones
> > > should be always attachable, unless they are special, e.g., notrace
> > > and stuff). We can easily check that by looking at the corresponding
> > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable

I'm still trying to build the kernel.. however ;-)

patch below adds the ftrace check only for static functions
and lets the externa go through.. but as you said, in this
case we'll need to figure out the 'notrace' and other checks
ftrace is doing

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index b124ec20a689..4d147406cfa5 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
 			continue;
 		if (!has_arg_names(cu, &fn->proto))
 			continue;
-		if (functions_cnt) {
+		if (!fn->external && functions_cnt) {
 			struct elf_function *func;
 			const char *name;
 
@@ -746,9 +746,6 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
 			if (!func || func->generated)
 				continue;
 			func->generated = true;
-		} else {
-			if (!fn->external)
-				continue;
 		}
 
 		btf_fnproto_id = btf_elf__add_func_proto(btfe, cu, &fn->proto, type_id_off);

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:

SNIP

> > > > >                 DW_AT_prototyped        (true)
> > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > >                 DW_AT_external  (true)
> > > > >
> > > > 
> > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > pahole ignores it. I don't know why this happens and it's quite
> > > > strange, given vfs_truncate is just a normal global function.
> > 
> > right, I can't see it in mcount adresses.. but it begins with instructions
> > that appears to be nops, which would suggest it's traceable
> > 
> > 	ffff80001031f430 <vfs_truncate>:
> > 	ffff80001031f430: 5f 24 03 d5   hint    #34
> > 	ffff80001031f434: 1f 20 03 d5   nop
> > 	ffff80001031f438: 1f 20 03 d5   nop
> > 	ffff80001031f43c: 3f 23 03 d5   hint    #25
> > 
> > > > 
> > > > I'd like to understand this issue before we try to fix it, but there
> > > > is at least one improvement we can make: pahole should check ftrace
> > > > addresses only for static functions, not the global ones (global ones
> > > > should be always attachable, unless they are special, e.g., notrace
> > > > and stuff). We can easily check that by looking at the corresponding
> > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> 
> I'm still trying to build the kernel.. however ;-)

I finally reproduced.. however arm's not using mcount_loc
but some other special section.. so it's new mess for me

however I tried the same build without LLVM=1 and it passed,
so my guess is that clang is doing something unexpected

jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Tue, Feb 09, 2021 at 05:13:38PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> 
> SNIP
> 
> > > > > >                 DW_AT_prototyped        (true)
> > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > >                 DW_AT_external  (true)
> > > > > >
> > > > > 
> > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > strange, given vfs_truncate is just a normal global function.
> > > 
> > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > that appears to be nops, which would suggest it's traceable
> > > 
> > > 	ffff80001031f430 <vfs_truncate>:
> > > 	ffff80001031f430: 5f 24 03 d5   hint    #34
> > > 	ffff80001031f434: 1f 20 03 d5   nop
> > > 	ffff80001031f438: 1f 20 03 d5   nop
> > > 	ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > 
> > > > > 
> > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > addresses only for static functions, not the global ones (global ones
> > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > 
> > I'm still trying to build the kernel.. however ;-)
> 
> I finally reproduced.. however arm's not using mcount_loc
> but some other special section.. so it's new mess for me
> 
> however I tried the same build without LLVM=1 and it passed,
> so my guess is that clang is doing something unexpected
> 
> jirka
> 

Additionally, if I remove the btfid generation section in
scripts/link-vmlinux.sh to bypass that and get a working Image.gz,
vfs_truncate is in the list of available functions:

/ # grep vfs_truncate /sys/kernel/debug/tracing/available_filter_functions
vfs_truncate

/ # cat /proc/version
Linux version 5.11.0-rc7-dirty (nathan@ubuntu-m3-large-x86) (ClangBuiltLinux clang version 13.0.0 (https://github.com/llvm/llvm-project 8f130f108fedfcf6cb80ef594560a87341028a37), LLD 13.0.0 (https://github.com/llvm/llvm-project 8f130f108fedfcf6cb80ef594560a87341028a37)) #1 SMP PREEMPT Tue Feb 9 09:25:36 MST 2021

/ # zcat /proc/config.gz | grep "DEBUG_INFO_BTF\|FTRACE\|BPF"
# CONFIG_CGROUP_BPF is not set
CONFIG_BPF=y
# CONFIG_BPF_LSM is not set
CONFIG_BPF_SYSCALL=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
# CONFIG_BPF_JIT_ALWAYS_ON is not set
CONFIG_BPF_JIT_DEFAULT_ON=y
# CONFIG_BPF_PRELOAD is not set
# CONFIG_NETFILTER_XT_MATCH_BPF is not set
# CONFIG_BPFILTER is not set
# CONFIG_NET_CLS_BPF is not set
# CONFIG_NET_ACT_BPF is not set
CONFIG_BPF_JIT=y
CONFIG_HAVE_EBPF_JIT=y
# CONFIG_PSTORE_FTRACE is not set
CONFIG_DEBUG_INFO_BTF=y
CONFIG_DEBUG_INFO_BTF_MODULES=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
# CONFIG_FTRACE_SYSCALLS is not set
CONFIG_BPF_EVENTS=y
CONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_RECORD_RECURSION is not set
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_TEST_BPF is not set

Cheers,
Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Sedat Dilek]

On Tue, Feb 9, 2021 at 5:35 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Tue, Feb 09, 2021 at 05:13:38PM +0100, Jiri Olsa wrote:
> > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> >
> > SNIP
> >
> > > > > > >                 DW_AT_prototyped        (true)
> > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > >                 DW_AT_external  (true)
> > > > > > >
> > > > > >
> > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > strange, given vfs_truncate is just a normal global function.
> > > >
> > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > that appears to be nops, which would suggest it's traceable
> > > >
> > > >   ffff80001031f430 <vfs_truncate>:
> > > >   ffff80001031f430: 5f 24 03 d5   hint    #34
> > > >   ffff80001031f434: 1f 20 03 d5   nop
> > > >   ffff80001031f438: 1f 20 03 d5   nop
> > > >   ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > >
> > > > > >
> > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > >
> > > I'm still trying to build the kernel.. however ;-)
> >
> > I finally reproduced.. however arm's not using mcount_loc
> > but some other special section.. so it's new mess for me
> >
> > however I tried the same build without LLVM=1 and it passed,
> > so my guess is that clang is doing something unexpected
> >
> > jirka
> >
>
> Additionally, if I remove the btfid generation section in
> scripts/link-vmlinux.sh to bypass that and get a working Image.gz,
> vfs_truncate is in the list of available functions:
>

I did the same trick by commenting in that section to provide a BROKEN
vmlinux file for linux-bpf folks.

[ BTFIDS-vmlinux.diff ]
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index eef40fa9485d..40f1b6aae553 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -330,7 +330,7 @@ vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
# fill in BTF IDs
if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
       info BTFIDS vmlinux
-       ${RESOLVE_BTFIDS} vmlinux
+       ##${RESOLVE_BTFIDS} vmlinux
fi

if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
- EOF -

We should ask linux-kbuild/Masahiro to have an option to OVERRIDE:
When scripts/link-vmlinux.sh fails all generated files like vmlinux get removed.
For further debugging they should be kept.

My €0.02.

- Sedat -

> / # grep vfs_truncate /sys/kernel/debug/tracing/available_filter_functions
> vfs_truncate
>
> / # cat /proc/version
> Linux version 5.11.0-rc7-dirty (nathan@ubuntu-m3-large-x86) (ClangBuiltLinux clang version 13.0.0 (https://github.com/llvm/llvm-project 8f130f108fedfcf6cb80ef594560a87341028a37), LLD 13.0.0 (https://github.com/llvm/llvm-project 8f130f108fedfcf6cb80ef594560a87341028a37)) #1 SMP PREEMPT Tue Feb 9 09:25:36 MST 2021
>
> / # zcat /proc/config.gz | grep "DEBUG_INFO_BTF\|FTRACE\|BPF"
> # CONFIG_CGROUP_BPF is not set
> CONFIG_BPF=y
> # CONFIG_BPF_LSM is not set
> CONFIG_BPF_SYSCALL=y
> CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
> # CONFIG_BPF_JIT_ALWAYS_ON is not set
> CONFIG_BPF_JIT_DEFAULT_ON=y
> # CONFIG_BPF_PRELOAD is not set
> # CONFIG_NETFILTER_XT_MATCH_BPF is not set
> # CONFIG_BPFILTER is not set
> # CONFIG_NET_CLS_BPF is not set
> # CONFIG_NET_ACT_BPF is not set
> CONFIG_BPF_JIT=y
> CONFIG_HAVE_EBPF_JIT=y
> # CONFIG_PSTORE_FTRACE is not set
> CONFIG_DEBUG_INFO_BTF=y
> CONFIG_DEBUG_INFO_BTF_MODULES=y
> CONFIG_HAVE_DYNAMIC_FTRACE=y
> CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
> CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
> CONFIG_FTRACE=y
> CONFIG_DYNAMIC_FTRACE=y
> CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
> # CONFIG_FTRACE_SYSCALLS is not set
> CONFIG_BPF_EVENTS=y
> CONFIG_FTRACE_MCOUNT_RECORD=y
> # CONFIG_FTRACE_RECORD_RECURSION is not set
> # CONFIG_FTRACE_STARTUP_TEST is not set
> # CONFIG_TEST_BPF is not set
>
> Cheers,
> Nathan
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210209163514.GA1226277%40ubuntu-m3-large-x86.

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nick Desaulniers]

On Tue, Feb 9, 2021 at 9:07 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> We should ask linux-kbuild/Masahiro to have an option to OVERRIDE:
> When scripts/link-vmlinux.sh fails all generated files like vmlinux get removed.
> For further debugging they should be kept.

I find it annoying, too.  But I usually just comment out the body of
cleanup() in scripts/link-vmlinux.sh.
-- 
Thanks,
~Nick Desaulniers

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Sedat Dilek]

On Tue, Feb 9, 2021 at 6:12 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Tue, Feb 9, 2021 at 9:07 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > We should ask linux-kbuild/Masahiro to have an option to OVERRIDE:
> > When scripts/link-vmlinux.sh fails all generated files like vmlinux get removed.
> > For further debugging they should be kept.
>
> I find it annoying, too.  But I usually just comment out the body of
> cleanup() in scripts/link-vmlinux.sh.
>

That's fine with me.

- Sedat -

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> 
> SNIP
> 
> > > > > >                 DW_AT_prototyped        (true)
> > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > >                 DW_AT_external  (true)
> > > > > >
> > > > > 
> > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > strange, given vfs_truncate is just a normal global function.
> > > 
> > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > that appears to be nops, which would suggest it's traceable
> > > 
> > > 	ffff80001031f430 <vfs_truncate>:
> > > 	ffff80001031f430: 5f 24 03 d5   hint    #34
> > > 	ffff80001031f434: 1f 20 03 d5   nop
> > > 	ffff80001031f438: 1f 20 03 d5   nop
> > > 	ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > 
> > > > > 
> > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > addresses only for static functions, not the global ones (global ones
> > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > 
> > I'm still trying to build the kernel.. however ;-)
> 
> I finally reproduced.. however arm's not using mcount_loc
> but some other special section.. so it's new mess for me

so ftrace data actualy has vfs_truncate address but with extra 4 bytes:

	ffff80001031f434

real vfs_truncate address:

	ffff80001031f430 g     F .text  0000000000000168 vfs_truncate

vfs_truncate disasm:

	ffff80001031f430 <vfs_truncate>:
	ffff80001031f430: 5f 24 03 d5   hint    #34
	ffff80001031f434: 1f 20 03 d5   nop
	ffff80001031f438: 1f 20 03 d5   nop
	ffff80001031f43c: 3f 23 03 d5   hint    #25

thats why we don't match it in pahole.. I checked few other functions
and some have the same problem and some match the function boundary

those that match don't have that first hint instrucion, like:

	ffff800010321e40 <do_faccessat>:
	ffff800010321e40: 1f 20 03 d5   nop
	ffff800010321e44: 1f 20 03 d5   nop
	ffff800010321e48: 3f 23 03 d5   hint    #25

any hints about hint instructions? ;-)

jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

adding Arnaldo to the loop

jirka

On Tue, Feb 09, 2021 at 08:06:16PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> > 
> > SNIP
> > 
> > > > > > >                 DW_AT_prototyped        (true)
> > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > >                 DW_AT_external  (true)
> > > > > > >
> > > > > > 
> > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > strange, given vfs_truncate is just a normal global function.
> > > > 
> > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > that appears to be nops, which would suggest it's traceable
> > > > 
> > > > 	ffff80001031f430 <vfs_truncate>:
> > > > 	ffff80001031f430: 5f 24 03 d5   hint    #34
> > > > 	ffff80001031f434: 1f 20 03 d5   nop
> > > > 	ffff80001031f438: 1f 20 03 d5   nop
> > > > 	ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > > 
> > > > > > 
> > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > > 
> > > I'm still trying to build the kernel.. however ;-)
> > 
> > I finally reproduced.. however arm's not using mcount_loc
> > but some other special section.. so it's new mess for me
> 
> so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
> 
> 	ffff80001031f434
> 
> real vfs_truncate address:
> 
> 	ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
> 
> vfs_truncate disasm:
> 
> 	ffff80001031f430 <vfs_truncate>:
> 	ffff80001031f430: 5f 24 03 d5   hint    #34
> 	ffff80001031f434: 1f 20 03 d5   nop
> 	ffff80001031f438: 1f 20 03 d5   nop
> 	ffff80001031f43c: 3f 23 03 d5   hint    #25
> 
> thats why we don't match it in pahole.. I checked few other functions
> and some have the same problem and some match the function boundary
> 
> those that match don't have that first hint instrucion, like:
> 
> 	ffff800010321e40 <do_faccessat>:
> 	ffff800010321e40: 1f 20 03 d5   nop
> 	ffff800010321e44: 1f 20 03 d5   nop
> 	ffff800010321e48: 3f 23 03 d5   hint    #25
> 
> any hints about hint instructions? ;-)
> 
> jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nick Desaulniers]

On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> >
> > SNIP
> >
> > > > > > >                 DW_AT_prototyped        (true)
> > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > >                 DW_AT_external  (true)
> > > > > > >
> > > > > >
> > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > strange, given vfs_truncate is just a normal global function.
> > > >
> > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > that appears to be nops, which would suggest it's traceable
> > > >
> > > >   ffff80001031f430 <vfs_truncate>:
> > > >   ffff80001031f430: 5f 24 03 d5   hint    #34
> > > >   ffff80001031f434: 1f 20 03 d5   nop
> > > >   ffff80001031f438: 1f 20 03 d5   nop
> > > >   ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > >
> > > > > >
> > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > >
> > > I'm still trying to build the kernel.. however ;-)
> >
> > I finally reproduced.. however arm's not using mcount_loc
> > but some other special section.. so it's new mess for me
>
> so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
>
>         ffff80001031f434
>
> real vfs_truncate address:
>
>         ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
>
> vfs_truncate disasm:
>
>         ffff80001031f430 <vfs_truncate>:
>         ffff80001031f430: 5f 24 03 d5   hint    #34
>         ffff80001031f434: 1f 20 03 d5   nop
>         ffff80001031f438: 1f 20 03 d5   nop
>         ffff80001031f43c: 3f 23 03 d5   hint    #25
>
> thats why we don't match it in pahole.. I checked few other functions
> and some have the same problem and some match the function boundary
>
> those that match don't have that first hint instrucion, like:
>
>         ffff800010321e40 <do_faccessat>:
>         ffff800010321e40: 1f 20 03 d5   nop
>         ffff800010321e44: 1f 20 03 d5   nop
>         ffff800010321e48: 3f 23 03 d5   hint    #25
>
> any hints about hint instructions? ;-)

aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
encoding space to make software backwards compatible on older hardware
that doesn't support such instructions.  Is this BTI, perhaps? (The
function is perhaps the destination of an indirect call?)
-- 
Thanks,
~Nick Desaulniers

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 12:09:31PM -0800, Nick Desaulniers wrote:
> On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> > > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> > >
> > > SNIP
> > >
> > > > > > > >                 DW_AT_prototyped        (true)
> > > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > > >                 DW_AT_external  (true)
> > > > > > > >
> > > > > > >
> > > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > > strange, given vfs_truncate is just a normal global function.
> > > > >
> > > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > > that appears to be nops, which would suggest it's traceable
> > > > >
> > > > >   ffff80001031f430 <vfs_truncate>:
> > > > >   ffff80001031f430: 5f 24 03 d5   hint    #34
> > > > >   ffff80001031f434: 1f 20 03 d5   nop
> > > > >   ffff80001031f438: 1f 20 03 d5   nop
> > > > >   ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > > >
> > > > > > >
> > > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > > >
> > > > I'm still trying to build the kernel.. however ;-)
> > >
> > > I finally reproduced.. however arm's not using mcount_loc
> > > but some other special section.. so it's new mess for me
> >
> > so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
> >
> >         ffff80001031f434
> >
> > real vfs_truncate address:
> >
> >         ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
> >
> > vfs_truncate disasm:
> >
> >         ffff80001031f430 <vfs_truncate>:
> >         ffff80001031f430: 5f 24 03 d5   hint    #34
> >         ffff80001031f434: 1f 20 03 d5   nop
> >         ffff80001031f438: 1f 20 03 d5   nop
> >         ffff80001031f43c: 3f 23 03 d5   hint    #25
> >
> > thats why we don't match it in pahole.. I checked few other functions
> > and some have the same problem and some match the function boundary
> >
> > those that match don't have that first hint instrucion, like:
> >
> >         ffff800010321e40 <do_faccessat>:
> >         ffff800010321e40: 1f 20 03 d5   nop
> >         ffff800010321e44: 1f 20 03 d5   nop
> >         ffff800010321e48: 3f 23 03 d5   hint    #25
> >
> > any hints about hint instructions? ;-)
> 
> aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
> encoding space to make software backwards compatible on older hardware
> that doesn't support such instructions.  Is this BTI, perhaps? (The
> function is perhaps the destination of an indirect call?)

I see, I think we can't take ftrace addresses as start of the function
because there could be extra instruction(s) related to the call before
it like here

we need to check ftrace address be within the function/symbol,
not exact start

jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Tue, Feb 9, 2021 at 7:09 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Feb 09, 2021 at 01:36:41PM +0100, Jiri Olsa wrote:
> > On Tue, Feb 09, 2021 at 12:49:04AM -0700, Nathan Chancellor wrote:
> > > On Mon, Feb 08, 2021 at 10:56:36PM -0800, Andrii Nakryiko wrote:
> > > > On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
> > > > <andrii.nakryiko@gmail.com> wrote:
> > > > >
> > > > > On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > >
> > > > > > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > >
> > > > > > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > > > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > > > > > >
> > > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > > > > > >
> > > > > > > > > $ scripts/config \
> > > > > > > > >     --file build/aarch64/.config \
> > > > > > > > >     -e BPF_SYSCALL \
> > > > > > > > >     -e DEBUG_INFO_BTF \
> > > > > > > > >     -e FTRACE \
> > > > > > > > >     -e FUNCTION_TRACER
> > > > > > > > >
> > > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > > > > > ...
> > > > > > > > > FAILED unresolved symbol vfs_truncate
> > > > > > > > > ...
> > > > > > > > >
> > > > > > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > > > > > although that seems obvious given that is what introduced
> > > > > > > > > BTF_ID(func, vfs_truncate).
> > > > > > > > >
> > > > > > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > > > > > have no idea what is going wrong here. Is this a known issue?
> > > > > > > > >
> > > > > > > >
> > > > > > > > I'll skip the reproduction games this time and will just request the
> > > > > > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > > > > > and see what's going on. Thanks.
> > > > > > > >
> > > > > > >
> > > > > > > Sure thing, let me know if this works. I uploaded in two places to make
> > > > > > > it easier to grab:
> > > > > > >
> > > > > > > zstd compressed:
> > > > > > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > > > > > >
> > > > > >
> > > > > > Thanks. I clearly see at least one instance of seemingly well-formed
> > > > > > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > > > > > it. Which means it should have been generated in BTF, but it doesn't
> > > > > > appear to be, so it does seem like a pahole bug. I (or someone else
> > > > > > before me) will continue tomorrow.
> > > > > >
> > > > > > $ llvm-dwarfdump vmlinux
> > > > > > ...
> > > > > >
> > > > > > 0x00052e6f:   DW_TAG_subprogram
> > > > > >                 DW_AT_name      ("vfs_truncate")
> > > > > >                 DW_AT_decl_file
> > > > > > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> > > > > >                 DW_AT_decl_line (2520)
> > > > > >                 DW_AT_prototyped        (true)
> > > > > >                 DW_AT_type      (0x000452cb "long int")
> > > > > >                 DW_AT_declaration       (true)
> > > > > >                 DW_AT_external  (true)
> > > > > >
> > > > > > 0x00052e7b:     DW_TAG_formal_parameter
> > > > > >                   DW_AT_type    (0x00045fc6 "const path*")
> > > > > >
> > > > > > 0x00052e80:     DW_TAG_formal_parameter
> > > > > >                   DW_AT_type    (0x00045213 "long long int")
> > > > > >
> > > > > > ...
> > > > > >
> > > > >
> > > > > ... and here's the *only* other one (not marked as declaration, but I
> > > > > thought we already handle that, Jiri?):
> > > > >
> > > > > 0x01d0da35:   DW_TAG_subprogram
> > > > >                 DW_AT_low_pc    (0xffff80001031f430)
> > > > >                 DW_AT_high_pc   (0xffff80001031f598)
> > > > >                 DW_AT_frame_base        (DW_OP_reg29)
> > > > >                 DW_AT_GNU_all_call_sites        (true)
> > > > >                 DW_AT_name      ("vfs_truncate")
> > > > >                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
> > > > >                 DW_AT_decl_line (69)
> > > > >                 DW_AT_prototyped        (true)
> > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > >                 DW_AT_external  (true)
> > > > >
> > > >
> > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > pahole ignores it. I don't know why this happens and it's quite
> > > > strange, given vfs_truncate is just a normal global function.
> >
> > right, I can't see it in mcount adresses.. but it begins with instructions
> > that appears to be nops, which would suggest it's traceable
> >
> >       ffff80001031f430 <vfs_truncate>:
> >       ffff80001031f430: 5f 24 03 d5   hint    #34
> >       ffff80001031f434: 1f 20 03 d5   nop
> >       ffff80001031f438: 1f 20 03 d5   nop
> >       ffff80001031f43c: 3f 23 03 d5   hint    #25
> >
> > > >
> > > > I'd like to understand this issue before we try to fix it, but there
> > > > is at least one improvement we can make: pahole should check ftrace
> > > > addresses only for static functions, not the global ones (global ones
> > > > should be always attachable, unless they are special, e.g., notrace
> > > > and stuff). We can easily check that by looking at the corresponding
> > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
>
> I'm still trying to build the kernel.. however ;-)
>
> patch below adds the ftrace check only for static functions
> and lets the externa go through.. but as you said, in this
> case we'll need to figure out the 'notrace' and other checks
> ftrace is doing
>
> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..4d147406cfa5 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
>                         continue;
>                 if (!has_arg_names(cu, &fn->proto))
>                         continue;
> -               if (functions_cnt) {
> +               if (!fn->external && functions_cnt) {

I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
FUNC ELF symbol be more reliable?

>                         struct elf_function *func;
>                         const char *name;
>
> @@ -746,9 +746,6 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
>                         if (!func || func->generated)
>                                 continue;
>                         func->generated = true;
> -               } else {
> -                       if (!fn->external)
> -                               continue;
>                 }
>
>                 btf_fnproto_id = btf_elf__add_func_proto(btfe, cu, &fn->proto, type_id_off);
>

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 09:50:48PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 12:09:31PM -0800, Nick Desaulniers wrote:
> > On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> > > > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> > > >
> > > > SNIP
> > > >
> > > > > > > > >                 DW_AT_prototyped        (true)
> > > > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > > > >                 DW_AT_external  (true)
> > > > > > > > >
> > > > > > > >
> > > > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > > > strange, given vfs_truncate is just a normal global function.
> > > > > >
> > > > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > > > that appears to be nops, which would suggest it's traceable
> > > > > >
> > > > > >   ffff80001031f430 <vfs_truncate>:
> > > > > >   ffff80001031f430: 5f 24 03 d5   hint    #34
> > > > > >   ffff80001031f434: 1f 20 03 d5   nop
> > > > > >   ffff80001031f438: 1f 20 03 d5   nop
> > > > > >   ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > > > >
> > > > > > > >
> > > > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > > > >
> > > > > I'm still trying to build the kernel.. however ;-)
> > > >
> > > > I finally reproduced.. however arm's not using mcount_loc
> > > > but some other special section.. so it's new mess for me
> > >
> > > so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
> > >
> > >         ffff80001031f434
> > >
> > > real vfs_truncate address:
> > >
> > >         ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
> > >
> > > vfs_truncate disasm:
> > >
> > >         ffff80001031f430 <vfs_truncate>:
> > >         ffff80001031f430: 5f 24 03 d5   hint    #34
> > >         ffff80001031f434: 1f 20 03 d5   nop
> > >         ffff80001031f438: 1f 20 03 d5   nop
> > >         ffff80001031f43c: 3f 23 03 d5   hint    #25
> > >
> > > thats why we don't match it in pahole.. I checked few other functions
> > > and some have the same problem and some match the function boundary
> > >
> > > those that match don't have that first hint instrucion, like:
> > >
> > >         ffff800010321e40 <do_faccessat>:
> > >         ffff800010321e40: 1f 20 03 d5   nop
> > >         ffff800010321e44: 1f 20 03 d5   nop
> > >         ffff800010321e48: 3f 23 03 d5   hint    #25
> > >
> > > any hints about hint instructions? ;-)
> > 
> > aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
> > encoding space to make software backwards compatible on older hardware
> > that doesn't support such instructions.  Is this BTI, perhaps? (The
> > function is perhaps the destination of an indirect call?)
> 
> I see, I think we can't take ftrace addresses as start of the function
> because there could be extra instruction(s) related to the call before
> it like here
> 
> we need to check ftrace address be within the function/symbol,
> not exact start

the build with gcc passed only because mcount data are all zeros
and pahole falls back to 'not-ftrace' mode

	$ llvm-objdump -t build/aarch64-gcc/vmlinux | grep mcount
	ffff800011eb4840 g       .init.data     0000000000000000 __stop_mcount_loc
	ffff800011e47d58 g       .init.data     0000000000000000 __start_mcount_loc

	$ llvm-objdump -s build/aarch64-gcc/vmlinux	
	ffff800011e47d50 00000000 00000000 00000000 00000000  ................
	ffff800011e47d60 00000000 00000000 00000000 00000000  ................
	ffff800011e47d70 00000000 00000000 00000000 00000000  ................
	ffff800011e47d80 00000000 00000000 00000000 00000000  ................
	ffff800011e47d90 00000000 00000000 00000000 00000000  ................
	...

	we should check on why it's zero

	Nathan, any chance you could run kernel built with gcc and check on ftrace?


the build with clang fails because the ftrace data are there,
but pahole takes them as 'start' of the function, which is wrong

	$ llvm-objdump -t build/aarch64/vmlinux | grep mcount
	ffff800011d27d10 g       .init.data     0000000000000000 __start_mcount_loc
	ffff800011d90038 g       .init.data     0000000000000000 __stop_mcount_loc

	$ llvm-objdump -s build/aarch64-gcc/vmlinux	
	ffff800011d27d10 cc330110 0080ffff 1c340110 0080ffff  .3.......4......
	ffff800011d27d20 6c340110 0080ffff 1004c111 0080ffff  l4..............
	ffff800011d27d30 3804c111 0080ffff 6004c111 0080ffff  8.......`.......
	ffff800011d27d40 8804c111 0080ffff 0405c111 0080ffff  ................
	ffff800011d27d50 3805c111 0080ffff 7c05c111 0080ffff  8.......|.......
	...

I think if we fix pahole to take check the ftrace address is
within the processed function, we should be fine.. I'll try to
send something soon

jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 12:59:51PM -0800, Andrii Nakryiko wrote:
> On Tue, Feb 9, 2021 at 7:09 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Feb 09, 2021 at 01:36:41PM +0100, Jiri Olsa wrote:
> > > On Tue, Feb 09, 2021 at 12:49:04AM -0700, Nathan Chancellor wrote:
> > > > On Mon, Feb 08, 2021 at 10:56:36PM -0800, Andrii Nakryiko wrote:
> > > > > On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
> > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > >
> > > > > > On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> > > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > > >
> > > > > > > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > > >
> > > > > > > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > > > > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > > > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > > > > > > >
> > > > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > > > > > > >
> > > > > > > > > > $ scripts/config \
> > > > > > > > > >     --file build/aarch64/.config \
> > > > > > > > > >     -e BPF_SYSCALL \
> > > > > > > > > >     -e DEBUG_INFO_BTF \
> > > > > > > > > >     -e FTRACE \
> > > > > > > > > >     -e FUNCTION_TRACER
> > > > > > > > > >
> > > > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > > > > > > ...
> > > > > > > > > > FAILED unresolved symbol vfs_truncate
> > > > > > > > > > ...
> > > > > > > > > >
> > > > > > > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > > > > > > although that seems obvious given that is what introduced
> > > > > > > > > > BTF_ID(func, vfs_truncate).
> > > > > > > > > >
> > > > > > > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > > > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > > > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > > > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > > > > > > have no idea what is going wrong here. Is this a known issue?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I'll skip the reproduction games this time and will just request the
> > > > > > > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > > > > > > and see what's going on. Thanks.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Sure thing, let me know if this works. I uploaded in two places to make
> > > > > > > > it easier to grab:
> > > > > > > >
> > > > > > > > zstd compressed:
> > > > > > > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > > > > > > >
> > > > > > >
> > > > > > > Thanks. I clearly see at least one instance of seemingly well-formed
> > > > > > > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > > > > > > it. Which means it should have been generated in BTF, but it doesn't
> > > > > > > appear to be, so it does seem like a pahole bug. I (or someone else
> > > > > > > before me) will continue tomorrow.
> > > > > > >
> > > > > > > $ llvm-dwarfdump vmlinux
> > > > > > > ...
> > > > > > >
> > > > > > > 0x00052e6f:   DW_TAG_subprogram
> > > > > > >                 DW_AT_name      ("vfs_truncate")
> > > > > > >                 DW_AT_decl_file
> > > > > > > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> > > > > > >                 DW_AT_decl_line (2520)
> > > > > > >                 DW_AT_prototyped        (true)
> > > > > > >                 DW_AT_type      (0x000452cb "long int")
> > > > > > >                 DW_AT_declaration       (true)
> > > > > > >                 DW_AT_external  (true)
> > > > > > >
> > > > > > > 0x00052e7b:     DW_TAG_formal_parameter
> > > > > > >                   DW_AT_type    (0x00045fc6 "const path*")
> > > > > > >
> > > > > > > 0x00052e80:     DW_TAG_formal_parameter
> > > > > > >                   DW_AT_type    (0x00045213 "long long int")
> > > > > > >
> > > > > > > ...
> > > > > > >
> > > > > >
> > > > > > ... and here's the *only* other one (not marked as declaration, but I
> > > > > > thought we already handle that, Jiri?):
> > > > > >
> > > > > > 0x01d0da35:   DW_TAG_subprogram
> > > > > >                 DW_AT_low_pc    (0xffff80001031f430)
> > > > > >                 DW_AT_high_pc   (0xffff80001031f598)
> > > > > >                 DW_AT_frame_base        (DW_OP_reg29)
> > > > > >                 DW_AT_GNU_all_call_sites        (true)
> > > > > >                 DW_AT_name      ("vfs_truncate")
> > > > > >                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
> > > > > >                 DW_AT_decl_line (69)
> > > > > >                 DW_AT_prototyped        (true)
> > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > >                 DW_AT_external  (true)
> > > > > >
> > > > >
> > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > strange, given vfs_truncate is just a normal global function.
> > >
> > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > that appears to be nops, which would suggest it's traceable
> > >
> > >       ffff80001031f430 <vfs_truncate>:
> > >       ffff80001031f430: 5f 24 03 d5   hint    #34
> > >       ffff80001031f434: 1f 20 03 d5   nop
> > >       ffff80001031f438: 1f 20 03 d5   nop
> > >       ffff80001031f43c: 3f 23 03 d5   hint    #25
> > >
> > > > >
> > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > addresses only for static functions, not the global ones (global ones
> > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> >
> > I'm still trying to build the kernel.. however ;-)
> >
> > patch below adds the ftrace check only for static functions
> > and lets the externa go through.. but as you said, in this
> > case we'll need to figure out the 'notrace' and other checks
> > ftrace is doing
> >
> > jirka
> >
> >
> > ---
> > diff --git a/btf_encoder.c b/btf_encoder.c
> > index b124ec20a689..4d147406cfa5 100644
> > --- a/btf_encoder.c
> > +++ b/btf_encoder.c
> > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> >                         continue;
> >                 if (!has_arg_names(cu, &fn->proto))
> >                         continue;
> > -               if (functions_cnt) {
> > +               if (!fn->external && functions_cnt) {
> 
> I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> FUNC ELF symbol be more reliable?

that'd mean extra bsearch on each processed function,
on the ther hand, we'are already slow ;-) I'll check
how big the slowdown would be

jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Tue, Feb 9, 2021 at 1:55 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Feb 09, 2021 at 12:59:51PM -0800, Andrii Nakryiko wrote:
> > On Tue, Feb 9, 2021 at 7:09 AM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Tue, Feb 09, 2021 at 01:36:41PM +0100, Jiri Olsa wrote:
> > > > On Tue, Feb 09, 2021 at 12:49:04AM -0700, Nathan Chancellor wrote:
> > > > > On Mon, Feb 08, 2021 at 10:56:36PM -0800, Andrii Nakryiko wrote:
> > > > > > On Mon, Feb 8, 2021 at 10:13 PM Andrii Nakryiko
> > > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > > >
> > > > > > > On Mon, Feb 8, 2021 at 10:09 PM Andrii Nakryiko
> > > > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > > > >
> > > > > > > > On Mon, Feb 8, 2021 at 9:23 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > > > >
> > > > > > > > > On Mon, Feb 08, 2021 at 08:45:43PM -0800, Andrii Nakryiko wrote:
> > > > > > > > > > On Mon, Feb 8, 2021 at 7:44 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > Recently, an issue with CONFIG_DEBUG_INFO_BTF was reported for arm64:
> > > > > > > > > > > https://groups.google.com/g/clang-built-linux/c/de_mNh23FOc/m/E7cu5BwbBAAJ
> > > > > > > > > > >
> > > > > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > > > > >                       LLVM=1 O=build/aarch64 defconfig
> > > > > > > > > > >
> > > > > > > > > > > $ scripts/config \
> > > > > > > > > > >     --file build/aarch64/.config \
> > > > > > > > > > >     -e BPF_SYSCALL \
> > > > > > > > > > >     -e DEBUG_INFO_BTF \
> > > > > > > > > > >     -e FTRACE \
> > > > > > > > > > >     -e FUNCTION_TRACER
> > > > > > > > > > >
> > > > > > > > > > > $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
> > > > > > > > > > >                       LLVM=1 O=build/aarch64 olddefconfig all
> > > > > > > > > > > ...
> > > > > > > > > > > FAILED unresolved symbol vfs_truncate
> > > > > > > > > > > ...
> > > > > > > > > > >
> > > > > > > > > > > My bisect landed on commit 6e22ab9da793 ("bpf: Add d_path helper")
> > > > > > > > > > > although that seems obvious given that is what introduced
> > > > > > > > > > > BTF_ID(func, vfs_truncate).
> > > > > > > > > > >
> > > > > > > > > > > I am using the latest pahole v1.20 and LLVM is at
> > > > > > > > > > > https://github.com/llvm/llvm-project/commit/14da287e18846ea86e45b421dc47f78ecc5aa7cb
> > > > > > > > > > > although I can reproduce back to LLVM 10.0.1, which is the earliest
> > > > > > > > > > > version that the kernel supports. I am very unfamiliar with BPF so I
> > > > > > > > > > > have no idea what is going wrong here. Is this a known issue?
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I'll skip the reproduction games this time and will just request the
> > > > > > > > > > vmlinux image. Please upload somewhere so that we can look at DWARF
> > > > > > > > > > and see what's going on. Thanks.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Sure thing, let me know if this works. I uploaded in two places to make
> > > > > > > > > it easier to grab:
> > > > > > > > >
> > > > > > > > > zstd compressed:
> > > > > > > > > https://github.com/nathanchance/bug-files/blob/3b2873751e29311e084ae2c71604a1963f5e1a48/btf-aarch64/vmlinux.zst
> > > > > > > > >
> > > > > > > >
> > > > > > > > Thanks. I clearly see at least one instance of seemingly well-formed
> > > > > > > > vfs_truncate DWARF declaration. Also there is a proper ELF symbol for
> > > > > > > > it. Which means it should have been generated in BTF, but it doesn't
> > > > > > > > appear to be, so it does seem like a pahole bug. I (or someone else
> > > > > > > > before me) will continue tomorrow.
> > > > > > > >
> > > > > > > > $ llvm-dwarfdump vmlinux
> > > > > > > > ...
> > > > > > > >
> > > > > > > > 0x00052e6f:   DW_TAG_subprogram
> > > > > > > >                 DW_AT_name      ("vfs_truncate")
> > > > > > > >                 DW_AT_decl_file
> > > > > > > > ("/home/nathan/cbl/src/linux/include/linux/fs.h")
> > > > > > > >                 DW_AT_decl_line (2520)
> > > > > > > >                 DW_AT_prototyped        (true)
> > > > > > > >                 DW_AT_type      (0x000452cb "long int")
> > > > > > > >                 DW_AT_declaration       (true)
> > > > > > > >                 DW_AT_external  (true)
> > > > > > > >
> > > > > > > > 0x00052e7b:     DW_TAG_formal_parameter
> > > > > > > >                   DW_AT_type    (0x00045fc6 "const path*")
> > > > > > > >
> > > > > > > > 0x00052e80:     DW_TAG_formal_parameter
> > > > > > > >                   DW_AT_type    (0x00045213 "long long int")
> > > > > > > >
> > > > > > > > ...
> > > > > > > >
> > > > > > >
> > > > > > > ... and here's the *only* other one (not marked as declaration, but I
> > > > > > > thought we already handle that, Jiri?):
> > > > > > >
> > > > > > > 0x01d0da35:   DW_TAG_subprogram
> > > > > > >                 DW_AT_low_pc    (0xffff80001031f430)
> > > > > > >                 DW_AT_high_pc   (0xffff80001031f598)
> > > > > > >                 DW_AT_frame_base        (DW_OP_reg29)
> > > > > > >                 DW_AT_GNU_all_call_sites        (true)
> > > > > > >                 DW_AT_name      ("vfs_truncate")
> > > > > > >                 DW_AT_decl_file ("/home/nathan/cbl/src/linux/fs/open.c")
> > > > > > >                 DW_AT_decl_line (69)
> > > > > > >                 DW_AT_prototyped        (true)
> > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > >                 DW_AT_external  (true)
> > > > > > >
> > > > > >
> > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > strange, given vfs_truncate is just a normal global function.
> > > >
> > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > that appears to be nops, which would suggest it's traceable
> > > >
> > > >       ffff80001031f430 <vfs_truncate>:
> > > >       ffff80001031f430: 5f 24 03 d5   hint    #34
> > > >       ffff80001031f434: 1f 20 03 d5   nop
> > > >       ffff80001031f438: 1f 20 03 d5   nop
> > > >       ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > >
> > > > > >
> > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > >
> > > I'm still trying to build the kernel.. however ;-)
> > >
> > > patch below adds the ftrace check only for static functions
> > > and lets the externa go through.. but as you said, in this
> > > case we'll need to figure out the 'notrace' and other checks
> > > ftrace is doing
> > >
> > > jirka
> > >
> > >
> > > ---
> > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > index b124ec20a689..4d147406cfa5 100644
> > > --- a/btf_encoder.c
> > > +++ b/btf_encoder.c
> > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > >                         continue;
> > >                 if (!has_arg_names(cu, &fn->proto))
> > >                         continue;
> > > -               if (functions_cnt) {
> > > +               if (!fn->external && functions_cnt) {
> >
> > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > FUNC ELF symbol be more reliable?
>
> that'd mean extra bsearch on each processed function,
> on the ther hand, we'are already slow ;-) I'll check
> how big the slowdown would be
>

We currently record addresses and do binary search. Now we need to
record address + size and still do binary search with a slightly
different semantics (find closest entry >= addr). Then just check that
it overlaps, taking into account the length of the function code. It
shouldn't result in a noticeable slowdown. Might be actually faster,
because we might avoid callback function call costs.

> jirka
>

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Tue, Feb 09, 2021 at 10:41:44PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 09:50:48PM +0100, Jiri Olsa wrote:
> > On Tue, Feb 09, 2021 at 12:09:31PM -0800, Nick Desaulniers wrote:
> > > On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa <jolsa@redhat.com> wrote:
> > > >
> > > > On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> > > > > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> > > > >
> > > > > SNIP
> > > > >
> > > > > > > > > >                 DW_AT_prototyped        (true)
> > > > > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > > > > >                 DW_AT_external  (true)
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > > > > strange, given vfs_truncate is just a normal global function.
> > > > > > >
> > > > > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > > > > that appears to be nops, which would suggest it's traceable
> > > > > > >
> > > > > > >   ffff80001031f430 <vfs_truncate>:
> > > > > > >   ffff80001031f430: 5f 24 03 d5   hint    #34
> > > > > > >   ffff80001031f434: 1f 20 03 d5   nop
> > > > > > >   ffff80001031f438: 1f 20 03 d5   nop
> > > > > > >   ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > > > > >
> > > > > > > > >
> > > > > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > > > > >
> > > > > > I'm still trying to build the kernel.. however ;-)
> > > > >
> > > > > I finally reproduced.. however arm's not using mcount_loc
> > > > > but some other special section.. so it's new mess for me
> > > >
> > > > so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
> > > >
> > > >         ffff80001031f434
> > > >
> > > > real vfs_truncate address:
> > > >
> > > >         ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
> > > >
> > > > vfs_truncate disasm:
> > > >
> > > >         ffff80001031f430 <vfs_truncate>:
> > > >         ffff80001031f430: 5f 24 03 d5   hint    #34
> > > >         ffff80001031f434: 1f 20 03 d5   nop
> > > >         ffff80001031f438: 1f 20 03 d5   nop
> > > >         ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > >
> > > > thats why we don't match it in pahole.. I checked few other functions
> > > > and some have the same problem and some match the function boundary
> > > >
> > > > those that match don't have that first hint instrucion, like:
> > > >
> > > >         ffff800010321e40 <do_faccessat>:
> > > >         ffff800010321e40: 1f 20 03 d5   nop
> > > >         ffff800010321e44: 1f 20 03 d5   nop
> > > >         ffff800010321e48: 3f 23 03 d5   hint    #25
> > > >
> > > > any hints about hint instructions? ;-)
> > > 
> > > aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
> > > encoding space to make software backwards compatible on older hardware
> > > that doesn't support such instructions.  Is this BTI, perhaps? (The
> > > function is perhaps the destination of an indirect call?)
> > 
> > I see, I think we can't take ftrace addresses as start of the function
> > because there could be extra instruction(s) related to the call before
> > it like here
> > 
> > we need to check ftrace address be within the function/symbol,
> > not exact start
> 
> the build with gcc passed only because mcount data are all zeros
> and pahole falls back to 'not-ftrace' mode
> 
> 	$ llvm-objdump -t build/aarch64-gcc/vmlinux | grep mcount
> 	ffff800011eb4840 g       .init.data     0000000000000000 __stop_mcount_loc
> 	ffff800011e47d58 g       .init.data     0000000000000000 __start_mcount_loc
> 
> 	$ llvm-objdump -s build/aarch64-gcc/vmlinux	
> 	ffff800011e47d50 00000000 00000000 00000000 00000000  ................
> 	ffff800011e47d60 00000000 00000000 00000000 00000000  ................
> 	ffff800011e47d70 00000000 00000000 00000000 00000000  ................
> 	ffff800011e47d80 00000000 00000000 00000000 00000000  ................
> 	ffff800011e47d90 00000000 00000000 00000000 00000000  ................
> 	...
> 
> 	we should check on why it's zero
> 
> 	Nathan, any chance you could run kernel built with gcc and check on ftrace?

Sure, with GCC 10.2.0:

/ # cat /proc/version
Linux version 5.11.0-rc7 (nathan@ubuntu-m3-large-x86) (aarch64-linux-gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.35) #1 SMP PREEMPT Tue Feb 9 16:04:19 MST 2021

/ # grep vfs_truncate /sys/kernel/debug/tracing/available_filter_functions
vfs_truncate

/ # zcat /proc/config.gz | grep "DEBUG_INFO_BTF\|FTRACE\|BPF"
# CONFIG_CGROUP_BPF is not set
CONFIG_BPF=y
# CONFIG_BPF_LSM is not set
CONFIG_BPF_SYSCALL=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
# CONFIG_BPF_JIT_ALWAYS_ON is not set
CONFIG_BPF_JIT_DEFAULT_ON=y
# CONFIG_BPF_PRELOAD is not set
# CONFIG_NETFILTER_XT_MATCH_BPF is not set
# CONFIG_BPFILTER is not set
# CONFIG_NET_CLS_BPF is not set
# CONFIG_NET_ACT_BPF is not set
CONFIG_BPF_JIT=y
CONFIG_HAVE_EBPF_JIT=y
# CONFIG_PSTORE_FTRACE is not set
CONFIG_DEBUG_INFO_BTF=y
CONFIG_DEBUG_INFO_BTF_MODULES=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
# CONFIG_FTRACE_SYSCALLS is not set
CONFIG_BPF_EVENTS=y
CONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_RECORD_RECURSION is not set
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_TEST_BPF is not set

Cheers,
Nathan

> the build with clang fails because the ftrace data are there,
> but pahole takes them as 'start' of the function, which is wrong
> 
> 	$ llvm-objdump -t build/aarch64/vmlinux | grep mcount
> 	ffff800011d27d10 g       .init.data     0000000000000000 __start_mcount_loc
> 	ffff800011d90038 g       .init.data     0000000000000000 __stop_mcount_loc
> 
> 	$ llvm-objdump -s build/aarch64-gcc/vmlinux	
> 	ffff800011d27d10 cc330110 0080ffff 1c340110 0080ffff  .3.......4......
> 	ffff800011d27d20 6c340110 0080ffff 1004c111 0080ffff  l4..............
> 	ffff800011d27d30 3804c111 0080ffff 6004c111 0080ffff  8.......`.......
> 	ffff800011d27d40 8804c111 0080ffff 0405c111 0080ffff  ................
> 	ffff800011d27d50 3805c111 0080ffff 7c05c111 0080ffff  8.......|.......
> 	...
> 
> I think if we fix pahole to take check the ftrace address is
> within the processed function, we should be fine.. I'll try to
> send something soon
> 
> jirka
> 

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Tue, Feb 09, 2021 at 12:09:31PM -0800, Nick Desaulniers wrote:
> On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
> > > On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
> > >
> > > SNIP
> > >
> > > > > > > >                 DW_AT_prototyped        (true)
> > > > > > > >                 DW_AT_type      (0x01cfdfe4 "long int")
> > > > > > > >                 DW_AT_external  (true)
> > > > > > > >
> > > > > > >
> > > > > > > Ok, the problem appears to be not in DWARF, but in mcount_loc data.
> > > > > > > vfs_truncate's address is not recorded as ftrace-attachable, and thus
> > > > > > > pahole ignores it. I don't know why this happens and it's quite
> > > > > > > strange, given vfs_truncate is just a normal global function.
> > > > >
> > > > > right, I can't see it in mcount adresses.. but it begins with instructions
> > > > > that appears to be nops, which would suggest it's traceable
> > > > >
> > > > >   ffff80001031f430 <vfs_truncate>:
> > > > >   ffff80001031f430: 5f 24 03 d5   hint    #34
> > > > >   ffff80001031f434: 1f 20 03 d5   nop
> > > > >   ffff80001031f438: 1f 20 03 d5   nop
> > > > >   ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > > >
> > > > > > >
> > > > > > > I'd like to understand this issue before we try to fix it, but there
> > > > > > > is at least one improvement we can make: pahole should check ftrace
> > > > > > > addresses only for static functions, not the global ones (global ones
> > > > > > > should be always attachable, unless they are special, e.g., notrace
> > > > > > > and stuff). We can easily check that by looking at the corresponding
> > > > > > > symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
> > > >
> > > > I'm still trying to build the kernel.. however ;-)
> > >
> > > I finally reproduced.. however arm's not using mcount_loc
> > > but some other special section.. so it's new mess for me
> >
> > so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
> >
> >         ffff80001031f434
> >
> > real vfs_truncate address:
> >
> >         ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
> >
> > vfs_truncate disasm:
> >
> >         ffff80001031f430 <vfs_truncate>:
> >         ffff80001031f430: 5f 24 03 d5   hint    #34
> >         ffff80001031f434: 1f 20 03 d5   nop
> >         ffff80001031f438: 1f 20 03 d5   nop
> >         ffff80001031f43c: 3f 23 03 d5   hint    #25
> >
> > thats why we don't match it in pahole.. I checked few other functions
> > and some have the same problem and some match the function boundary
> >
> > those that match don't have that first hint instrucion, like:
> >
> >         ffff800010321e40 <do_faccessat>:
> >         ffff800010321e40: 1f 20 03 d5   nop
> >         ffff800010321e44: 1f 20 03 d5   nop
> >         ffff800010321e48: 3f 23 03 d5   hint    #25
> >
> > any hints about hint instructions? ;-)
> 
> aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
> encoding space to make software backwards compatible on older hardware
> that doesn't support such instructions.  Is this BTI, perhaps? (The
> function is perhaps the destination of an indirect call?)

It seems like it. The issue is not reproducible when
CONFIG_ARM64_BTI_KERNEL is not set.

Cheers,
Nathan

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Daniel Kiss]

> On 10 Feb 2021, at 01:02, Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Tue, Feb 09, 2021 at 12:09:31PM -0800, Nick Desaulniers wrote:
>> On Tue, Feb 9, 2021 at 11:06 AM Jiri Olsa <jolsa@redhat.com> wrote:
>>>
>>> On Tue, Feb 09, 2021 at 05:13:42PM +0100, Jiri Olsa wrote:
>>>> On Tue, Feb 09, 2021 at 04:09:36PM +0100, Jiri Olsa wrote:
>>>>
>>>> SNIP
>>>>
>>>>>>>>>                DW_AT_prototyped        (true)
>>>>>>>>>                DW_AT_type      (0x01cfdfe4 "long int")
>>>>>>>>>                DW_AT_external  (true)
>>>>>>>>>
>>>>>>>>
>>>>>>>> Ok, the problem appears to be not in DWARF, but in mcount_loc data.
>>>>>>>> vfs_truncate's address is not recorded as ftrace-attachable, and thus
>>>>>>>> pahole ignores it. I don't know why this happens and it's quite
>>>>>>>> strange, given vfs_truncate is just a normal global function.
>>>>>>
>>>>>> right, I can't see it in mcount adresses.. but it begins with instructions
>>>>>> that appears to be nops, which would suggest it's traceable
>>>>>>
>>>>>>  ffff80001031f430 <vfs_truncate>:
>>>>>>  ffff80001031f430: 5f 24 03 d5   hint    #34
>>>>>>  ffff80001031f434: 1f 20 03 d5   nop
>>>>>>  ffff80001031f438: 1f 20 03 d5   nop
>>>>>>  ffff80001031f43c: 3f 23 03 d5   hint    #25
>>>>>>
>>>>>>>>
>>>>>>>> I'd like to understand this issue before we try to fix it, but there
>>>>>>>> is at least one improvement we can make: pahole should check ftrace
>>>>>>>> addresses only for static functions, not the global ones (global ones
>>>>>>>> should be always attachable, unless they are special, e.g., notrace
>>>>>>>> and stuff). We can easily check that by looking at the corresponding
>>>>>>>> symbol. But I'd like to verify that vfs_truncate is ftrace-attachable
>>>>>
>>>>> I'm still trying to build the kernel.. however ;-)
>>>>
>>>> I finally reproduced.. however arm's not using mcount_loc
>>>> but some other special section.. so it's new mess for me
>>>
>>> so ftrace data actualy has vfs_truncate address but with extra 4 bytes:
>>>
>>>        ffff80001031f434
>>>
>>> real vfs_truncate address:
>>>
>>>        ffff80001031f430 g     F .text  0000000000000168 vfs_truncate
>>>
>>> vfs_truncate disasm:
>>>
>>>        ffff80001031f430 <vfs_truncate>:
>>>        ffff80001031f430: 5f 24 03 d5   hint    #34
>>>        ffff80001031f434: 1f 20 03 d5   nop
>>>        ffff80001031f438: 1f 20 03 d5   nop
>>>        ffff80001031f43c: 3f 23 03 d5   hint    #25
>>>
>>> thats why we don't match it in pahole.. I checked few other functions
>>> and some have the same problem and some match the function boundary
>>>
>>> those that match don't have that first hint instrucion, like:
>>>
>>>        ffff800010321e40 <do_faccessat>:
>>>        ffff800010321e40: 1f 20 03 d5   nop
>>>        ffff800010321e44: 1f 20 03 d5   nop
>>>        ffff800010321e48: 3f 23 03 d5   hint    #25
>>>
>>> any hints about hint instructions? ;-)
>>
>> aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
>> encoding space to make software backwards compatible on older hardware
>> that doesn't support such instructions.  Is this BTI, perhaps? (The
>> function is perhaps the destination of an indirect call?)
>
> It seems like it. The issue is not reproducible when
> CONFIG_ARM64_BTI_KERNEL is not set.
>
llvm-objdump --mattr=pa —mattr=bti -d … will print new mnemonic for the hint space instructions.

It is intentional to put a landing pad (BTI) at the very beginning for the patchable functions.
       ffff80001031f430 <vfs_truncate>:
       ffff80001031f430: 5f 24 03 d5   hint    #34 // BTI C <— landing pad for indirect calls.
       ffff80001031f434: 1f 20 03 d5   nop   // <— the to be patched area.
       ffff80001031f438: 1f 20 03 d5   nop
       ffff80001031f43c: 3f 23 03 d5   hint    #25 // paciasp <— protecting the link register..
The ftrace data(ffff80001031f434) seems correct because it should point to the “nops".

HTH

Cheers,
Daniel

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

RE: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[David Laight]

> > > vfs_truncate disasm:
> > >
> > >         ffff80001031f430 <vfs_truncate>:
> > >         ffff80001031f430: 5f 24 03 d5   hint    #34
> > >         ffff80001031f434: 1f 20 03 d5   nop
> > >         ffff80001031f438: 1f 20 03 d5   nop
> > >         ffff80001031f43c: 3f 23 03 d5   hint    #25
> > >
> > > thats why we don't match it in pahole.. I checked few other functions
> > > and some have the same problem and some match the function boundary
> > >
> > > those that match don't have that first hint instrucion, like:
> > >
> > >         ffff800010321e40 <do_faccessat>:
> > >         ffff800010321e40: 1f 20 03 d5   nop
> > >         ffff800010321e44: 1f 20 03 d5   nop
> > >         ffff800010321e48: 3f 23 03 d5   hint    #25
> > >
> > > any hints about hint instructions? ;-)
> >
> > aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
> > encoding space to make software backwards compatible on older hardware
> > that doesn't support such instructions.  Is this BTI, perhaps? (The
> > function is perhaps the destination of an indirect call?)
> 
> It seems like it. The issue is not reproducible when
> CONFIG_ARM64_BTI_KERNEL is not set.

Is the compiler/linker doing something 'crazy'?

If a function address is taken then the BTI instruction is placed
before the function body and the symbol moved.
But non-indirect calls still jump to the original start of the function.
(In this case the first nop.)

This saves the execution time of the BTI instruction for non-indirect
calls.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Wed, Feb 10, 2021 at 11:34:25AM +0000, David Laight wrote:
> > > > vfs_truncate disasm:
> > > >
> > > >         ffff80001031f430 <vfs_truncate>:
> > > >         ffff80001031f430: 5f 24 03 d5   hint    #34
> > > >         ffff80001031f434: 1f 20 03 d5   nop
> > > >         ffff80001031f438: 1f 20 03 d5   nop
> > > >         ffff80001031f43c: 3f 23 03 d5   hint    #25
> > > >
> > > > thats why we don't match it in pahole.. I checked few other functions
> > > > and some have the same problem and some match the function boundary
> > > >
> > > > those that match don't have that first hint instrucion, like:
> > > >
> > > >         ffff800010321e40 <do_faccessat>:
> > > >         ffff800010321e40: 1f 20 03 d5   nop
> > > >         ffff800010321e44: 1f 20 03 d5   nop
> > > >         ffff800010321e48: 3f 23 03 d5   hint    #25
> > > >
> > > > any hints about hint instructions? ;-)
> > >
> > > aarch64 makes *some* newer instructions reuse the "hint" ie "nop"
> > > encoding space to make software backwards compatible on older hardware
> > > that doesn't support such instructions.  Is this BTI, perhaps? (The
> > > function is perhaps the destination of an indirect call?)
> > 
> > It seems like it. The issue is not reproducible when
> > CONFIG_ARM64_BTI_KERNEL is not set.
> 
> Is the compiler/linker doing something 'crazy'?

I need to check the gcc build, where pahole in my test sees
zero ftrace addresses, while booted kernel shows ftrace working

> 
> If a function address is taken then the BTI instruction is placed
> before the function body and the symbol moved.
> But non-indirect calls still jump to the original start of the function.
> (In this case the first nop.)

for clang build, it looks like pahole needs fix not to assume that
ftrace address match with function start, I'll send it later today

thanks,
jirka

> 
> This saves the execution time of the BTI instruction for non-indirect
> calls.
> 
> 	David
> 
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
> 

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Tue, Feb 09, 2021 at 02:00:29PM -0800, Andrii Nakryiko wrote:

SNIP

> > > > I'm still trying to build the kernel.. however ;-)
> > > >
> > > > patch below adds the ftrace check only for static functions
> > > > and lets the externa go through.. but as you said, in this
> > > > case we'll need to figure out the 'notrace' and other checks
> > > > ftrace is doing
> > > >
> > > > jirka
> > > >
> > > >
> > > > ---
> > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > index b124ec20a689..4d147406cfa5 100644
> > > > --- a/btf_encoder.c
> > > > +++ b/btf_encoder.c
> > > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > > >                         continue;
> > > >                 if (!has_arg_names(cu, &fn->proto))
> > > >                         continue;
> > > > -               if (functions_cnt) {
> > > > +               if (!fn->external && functions_cnt) {
> > >
> > > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > > FUNC ELF symbol be more reliable?
> >
> > that'd mean extra bsearch on each processed function,
> > on the ther hand, we'are already slow ;-) I'll check
> > how big the slowdown would be
> >
> 
> We currently record addresses and do binary search. Now we need to
> record address + size and still do binary search with a slightly
> different semantics (find closest entry >= addr). Then just check that
> it overlaps, taking into account the length of the function code. It
> shouldn't result in a noticeable slowdown. Might be actually faster,
> because we might avoid callback function call costs.

I'm still not sure how to handle the external check for function via elf,
but below is change for checking that ftrace addrs are within elf functions

seems to work in my tests, I'll run some more tests and send full patch

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index b124ec20a689..548a12847f99 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -36,6 +36,7 @@ struct funcs_layout {
 struct elf_function {
 	const char	*name;
 	unsigned long	 addr;
+	unsigned long	 end;
 	unsigned long	 sh_addr;
 	bool		 generated;
 };
@@ -44,7 +45,7 @@ static struct elf_function *functions;
 static int functions_alloc;
 static int functions_cnt;
 
-static int functions_cmp(const void *_a, const void *_b)
+static int functions_cmp_name(const void *_a, const void *_b)
 {
 	const struct elf_function *a = _a;
 	const struct elf_function *b = _b;
@@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
 	return strcmp(a->name, b->name);
 }
 
+static int functions_cmp_addr(const void *_a, const void *_b)
+{
+	const struct elf_function *a = _a;
+	const struct elf_function *b = _b;
+
+	if (a->addr == b->addr)
+		return 0;
+	return a->addr < b->addr ? -1 : 1;
+}
+
 static void delete_functions(void)
 {
 	free(functions);
@@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
 
 	functions[functions_cnt].name = name;
 	functions[functions_cnt].addr = elf_sym__value(sym);
+	functions[functions_cnt].end = (__u64) -1;
 	functions[functions_cnt].sh_addr = sh.sh_addr;
 	functions[functions_cnt].generated = false;
 	functions_cnt++;
@@ -236,9 +248,25 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
 	return 0;
 }
 
+static bool is_addr_in_func(__u64 addr, struct elf_function *func, bool kmod)
+{
+	/*
+	 * For vmlinux image both addrs[x] and functions[x]::addr
+	 * values are final address and are comparable.
+	 *
+	 * For kernel module addrs[x] is final address, but
+	 * functions[x]::addr is relative address within section
+	 * and needs to be relocated by adding sh_addr.
+	 */
+	__u64 start = kmod ? func->addr + func->sh_addr : func->addr;
+	__u64 end = kmod ? func->end+ func->sh_addr : func->end;
+
+	return start <= addr && addr < end;
+}
+
 static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 {
-	__u64 *addrs, count, i;
+	__u64 *addrs, count, i_func, i_addr;
 	int functions_valid = 0;
 	bool kmod = false;
 
@@ -266,43 +294,62 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 		return 0;
 	}
 
-	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
-	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
-
 	/*
-	 * Let's got through all collected functions and filter
-	 * out those that are not in ftrace.
+	 * Sort both functions and addrs so we can iterate
+	 * both of them simultaneously and found matching
+	 * func/addr pairs.
 	 */
-	for (i = 0; i < functions_cnt; i++) {
-		struct elf_function *func = &functions[i];
-		/*
-		 * For vmlinux image both addrs[x] and functions[x]::addr
-		 * values are final address and are comparable.
-		 *
-		 * For kernel module addrs[x] is final address, but
-		 * functions[x]::addr is relative address within section
-		 * and needs to be relocated by adding sh_addr.
-		 */
-		__u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
+	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
+	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
+
+	for (i_func = 0, i_addr = 0; i_func < functions_cnt; i_func++) {
+		struct elf_function *func = &functions[i_func];
+
+		if (i_func + 1 < functions_cnt)
+			func->end = functions[i_func + 1].addr;
+
+		for (; i_addr < count; i_addr++) {
+			__u64 addr = addrs[i_addr];
+
+			/* Functions are  ahead, catch up with addrs. */
+			if (addr < func->addr)
+				continue;
+
+			/* Addr is within function - mark function as valid. */
+			if (is_addr_in_func(addr, func, kmod)) {
+				/*
+				 * We iterate over sorted array, so we can easily skip
+				 * not valid item and move following valid field into
+				 * its place, and still keep the 'new' array sorted.
+				 */
+				if (i_func != functions_valid)
+					functions[functions_valid] = functions[i_func];
+				functions_valid++;
+				i_addr++;
+			}
 
-		/* Make sure function is within ftrace addresses. */
-		if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
 			/*
-			 * We iterate over sorted array, so we can easily skip
-			 * not valid item and move following valid field into
-			 * its place, and still keep the 'new' array sorted.
+			 * Addrs are ahead, catch up with functions, or we just
+			 * found valid function and want to move to another.
 			 */
-			if (i != functions_valid)
-				functions[functions_valid] = functions[i];
-			functions_valid++;
+			break;
 		}
 	}
 
+	if (btf_elf__verbose) {
+		printf("Found %d functions out of %d symbols and %llu ftrace addresses.\n",
+			functions_valid, functions_cnt, count);
+	}
+
 	functions_cnt = functions_valid;
 	free(addrs);
 
-	if (btf_elf__verbose)
-		printf("Found %d functions!\n", functions_cnt);
+	/*
+	 * And finaly sort 'valid' functions by name,
+	 * so find_function can be used.
+	 */
+	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
+
 	return 0;
 }
 
@@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
 	struct elf_function key = { .name = name };
 
 	return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
-		       functions_cmp);
+		       functions_cmp_name);
 }
 
 static bool btf_name_char_ok(char c, bool first)

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Wed, Feb 10, 2021 at 02:26:43PM +0100, Jiri Olsa wrote:
> On Tue, Feb 09, 2021 at 02:00:29PM -0800, Andrii Nakryiko wrote:
> 
> SNIP
> 
> > > > > I'm still trying to build the kernel.. however ;-)
> > > > >
> > > > > patch below adds the ftrace check only for static functions
> > > > > and lets the externa go through.. but as you said, in this
> > > > > case we'll need to figure out the 'notrace' and other checks
> > > > > ftrace is doing
> > > > >
> > > > > jirka
> > > > >
> > > > >
> > > > > ---
> > > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > > index b124ec20a689..4d147406cfa5 100644
> > > > > --- a/btf_encoder.c
> > > > > +++ b/btf_encoder.c
> > > > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > > > >                         continue;
> > > > >                 if (!has_arg_names(cu, &fn->proto))
> > > > >                         continue;
> > > > > -               if (functions_cnt) {
> > > > > +               if (!fn->external && functions_cnt) {
> > > >
> > > > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > > > FUNC ELF symbol be more reliable?
> > >
> > > that'd mean extra bsearch on each processed function,
> > > on the ther hand, we'are already slow ;-) I'll check
> > > how big the slowdown would be
> > >
> > 
> > We currently record addresses and do binary search. Now we need to
> > record address + size and still do binary search with a slightly
> > different semantics (find closest entry >= addr). Then just check that
> > it overlaps, taking into account the length of the function code. It
> > shouldn't result in a noticeable slowdown. Might be actually faster,
> > because we might avoid callback function call costs.
> 
> I'm still not sure how to handle the external check for function via elf,
> but below is change for checking that ftrace addrs are within elf functions
> 
> seems to work in my tests, I'll run some more tests and send full patch
> 
> jirka

This passes the build test with the initial problematic configuration
file and the reduced one that I came up with. That is about the extent
to which I can verify that it works though :)

> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..548a12847f99 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -36,6 +36,7 @@ struct funcs_layout {
>  struct elf_function {
>  	const char	*name;
>  	unsigned long	 addr;
> +	unsigned long	 end;
>  	unsigned long	 sh_addr;
>  	bool		 generated;
>  };
> @@ -44,7 +45,7 @@ static struct elf_function *functions;
>  static int functions_alloc;
>  static int functions_cnt;
>  
> -static int functions_cmp(const void *_a, const void *_b)
> +static int functions_cmp_name(const void *_a, const void *_b)
>  {
>  	const struct elf_function *a = _a;
>  	const struct elf_function *b = _b;
> @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
>  	return strcmp(a->name, b->name);
>  }
>  
> +static int functions_cmp_addr(const void *_a, const void *_b)
> +{
> +	const struct elf_function *a = _a;
> +	const struct elf_function *b = _b;
> +
> +	if (a->addr == b->addr)
> +		return 0;
> +	return a->addr < b->addr ? -1 : 1;
> +}
> +
>  static void delete_functions(void)
>  {
>  	free(functions);
> @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
>  
>  	functions[functions_cnt].name = name;
>  	functions[functions_cnt].addr = elf_sym__value(sym);
> +	functions[functions_cnt].end = (__u64) -1;
>  	functions[functions_cnt].sh_addr = sh.sh_addr;
>  	functions[functions_cnt].generated = false;
>  	functions_cnt++;
> @@ -236,9 +248,25 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
>  	return 0;
>  }
>  
> +static bool is_addr_in_func(__u64 addr, struct elf_function *func, bool kmod)
> +{
> +	/*
> +	 * For vmlinux image both addrs[x] and functions[x]::addr
> +	 * values are final address and are comparable.
> +	 *
> +	 * For kernel module addrs[x] is final address, but
> +	 * functions[x]::addr is relative address within section
> +	 * and needs to be relocated by adding sh_addr.
> +	 */
> +	__u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> +	__u64 end = kmod ? func->end+ func->sh_addr : func->end;
> +
> +	return start <= addr && addr < end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
> -	__u64 *addrs, count, i;
> +	__u64 *addrs, count, i_func, i_addr;
>  	int functions_valid = 0;
>  	bool kmod = false;
>  
> @@ -266,43 +294,62 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  		return 0;
>  	}
>  
> -	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> -	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> -
>  	/*
> -	 * Let's got through all collected functions and filter
> -	 * out those that are not in ftrace.
> +	 * Sort both functions and addrs so we can iterate
> +	 * both of them simultaneously and found matching
> +	 * func/addr pairs.
>  	 */
> -	for (i = 0; i < functions_cnt; i++) {
> -		struct elf_function *func = &functions[i];
> -		/*
> -		 * For vmlinux image both addrs[x] and functions[x]::addr
> -		 * values are final address and are comparable.
> -		 *
> -		 * For kernel module addrs[x] is final address, but
> -		 * functions[x]::addr is relative address within section
> -		 * and needs to be relocated by adding sh_addr.
> -		 */
> -		__u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> +	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> +	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
> +
> +	for (i_func = 0, i_addr = 0; i_func < functions_cnt; i_func++) {
> +		struct elf_function *func = &functions[i_func];
> +
> +		if (i_func + 1 < functions_cnt)
> +			func->end = functions[i_func + 1].addr;
> +
> +		for (; i_addr < count; i_addr++) {
> +			__u64 addr = addrs[i_addr];
> +
> +			/* Functions are  ahead, catch up with addrs. */
> +			if (addr < func->addr)
> +				continue;
> +
> +			/* Addr is within function - mark function as valid. */
> +			if (is_addr_in_func(addr, func, kmod)) {
> +				/*
> +				 * We iterate over sorted array, so we can easily skip
> +				 * not valid item and move following valid field into
> +				 * its place, and still keep the 'new' array sorted.
> +				 */
> +				if (i_func != functions_valid)
> +					functions[functions_valid] = functions[i_func];
> +				functions_valid++;
> +				i_addr++;
> +			}
>  
> -		/* Make sure function is within ftrace addresses. */
> -		if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
>  			/*
> -			 * We iterate over sorted array, so we can easily skip
> -			 * not valid item and move following valid field into
> -			 * its place, and still keep the 'new' array sorted.
> +			 * Addrs are ahead, catch up with functions, or we just
> +			 * found valid function and want to move to another.
>  			 */
> -			if (i != functions_valid)
> -				functions[functions_valid] = functions[i];
> -			functions_valid++;
> +			break;
>  		}
>  	}
>  
> +	if (btf_elf__verbose) {
> +		printf("Found %d functions out of %d symbols and %llu ftrace addresses.\n",
> +			functions_valid, functions_cnt, count);
> +	}
> +
>  	functions_cnt = functions_valid;
>  	free(addrs);
>  
> -	if (btf_elf__verbose)
> -		printf("Found %d functions!\n", functions_cnt);
> +	/*
> +	 * And finaly sort 'valid' functions by name,
> +	 * so find_function can be used.
> +	 */
> +	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
> +
>  	return 0;
>  }
>  
> @@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
>  	struct elf_function key = { .name = name };
>  
>  	return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> -		       functions_cmp);
> +		       functions_cmp_name);
>  }
>  
>  static bool btf_name_char_ok(char c, bool first)
> 

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Wed, Feb 10, 2021 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Feb 09, 2021 at 02:00:29PM -0800, Andrii Nakryiko wrote:
>
> SNIP
>
> > > > > I'm still trying to build the kernel.. however ;-)
> > > > >
> > > > > patch below adds the ftrace check only for static functions
> > > > > and lets the externa go through.. but as you said, in this
> > > > > case we'll need to figure out the 'notrace' and other checks
> > > > > ftrace is doing
> > > > >
> > > > > jirka
> > > > >
> > > > >
> > > > > ---
> > > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > > index b124ec20a689..4d147406cfa5 100644
> > > > > --- a/btf_encoder.c
> > > > > +++ b/btf_encoder.c
> > > > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > > > >                         continue;
> > > > >                 if (!has_arg_names(cu, &fn->proto))
> > > > >                         continue;
> > > > > -               if (functions_cnt) {
> > > > > +               if (!fn->external && functions_cnt) {
> > > >
> > > > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > > > FUNC ELF symbol be more reliable?
> > >
> > > that'd mean extra bsearch on each processed function,
> > > on the ther hand, we'are already slow ;-) I'll check
> > > how big the slowdown would be
> > >
> >
> > We currently record addresses and do binary search. Now we need to
> > record address + size and still do binary search with a slightly
> > different semantics (find closest entry >= addr). Then just check that
> > it overlaps, taking into account the length of the function code. It
> > shouldn't result in a noticeable slowdown. Might be actually faster,
> > because we might avoid callback function call costs.
>
> I'm still not sure how to handle the external check for function via elf,

I might be missing something, but don't all functions have
corresponding ELF symbols? And then symbol can have LOCAL or GLOBAL
type. LOCALs are supposed to be not visible outside respective CUs (so
correspond to static functions), while GLOBALs are extern-able funcs.
So if func's symbol is GLOBAL, it should be ok to assume it's
attachable (not inlined, at least).

> but below is change for checking that ftrace addrs are within elf functions
>
> seems to work in my tests, I'll run some more tests and send full patch

It seems unnecessarily convoluted. I was thinking about something like
this (the diff will totally be screwed up by gmail, and I haven't even
compiled it):

diff --git a/btf_encoder.c b/btf_encoder.c
index b124ec20a689..8162b238bd43 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
**paddrs, __u64 *pcount)
        return 0;
 }

+struct func_seg { __u64 start; __u64 end; };
+
+static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
+{
+       size_t l = 0, r = len - 1, m;
+
+       while (l < r) {
+               m = l + (r - l + 1) / 2;
+               if (segs[m].start <= addr)
+                       l = m;
+               else
+                       r = m - 1;
+       }
+
+       return segs[l].start <= addr && addr < segs[l].end;
+}
+
 static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 {
        __u64 *addrs, count, i;
@@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
struct funcs_layout *fl)
                __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;

                /* Make sure function is within ftrace addresses. */
-               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
+               if (func_exists(addrs, count, addr))
                        /*
                         * We iterate over sorted array, so we can easily skip
                         * not valid item and move following valid field into


So the idea is to use address segments and check whether there is a
segment that overlaps with a given address by first binary searching
for a segment with the largest starting address that is <= addr. And
then just confirming that segment does overlap with the requested
address.

WDYT?

>
> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..548a12847f99 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -36,6 +36,7 @@ struct funcs_layout {
>  struct elf_function {
>         const char      *name;
>         unsigned long    addr;
> +       unsigned long    end;
>         unsigned long    sh_addr;
>         bool             generated;
>  };
> @@ -44,7 +45,7 @@ static struct elf_function *functions;
>  static int functions_alloc;
>  static int functions_cnt;
>
> -static int functions_cmp(const void *_a, const void *_b)
> +static int functions_cmp_name(const void *_a, const void *_b)
>  {
>         const struct elf_function *a = _a;
>         const struct elf_function *b = _b;
> @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
>         return strcmp(a->name, b->name);
>  }
>
> +static int functions_cmp_addr(const void *_a, const void *_b)
> +{
> +       const struct elf_function *a = _a;
> +       const struct elf_function *b = _b;
> +
> +       if (a->addr == b->addr)
> +               return 0;
> +       return a->addr < b->addr ? -1 : 1;
> +}
> +
>  static void delete_functions(void)
>  {
>         free(functions);
> @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
>
>         functions[functions_cnt].name = name;
>         functions[functions_cnt].addr = elf_sym__value(sym);
> +       functions[functions_cnt].end = (__u64) -1;
>         functions[functions_cnt].sh_addr = sh.sh_addr;
>         functions[functions_cnt].generated = false;
>         functions_cnt++;
> @@ -236,9 +248,25 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
>         return 0;
>  }
>
> +static bool is_addr_in_func(__u64 addr, struct elf_function *func, bool kmod)
> +{
> +       /*
> +        * For vmlinux image both addrs[x] and functions[x]::addr
> +        * values are final address and are comparable.
> +        *
> +        * For kernel module addrs[x] is final address, but
> +        * functions[x]::addr is relative address within section
> +        * and needs to be relocated by adding sh_addr.
> +        */
> +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> +       __u64 end = kmod ? func->end+ func->sh_addr : func->end;
> +
> +       return start <= addr && addr < end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
> -       __u64 *addrs, count, i;
> +       __u64 *addrs, count, i_func, i_addr;
>         int functions_valid = 0;
>         bool kmod = false;
>
> @@ -266,43 +294,62 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>                 return 0;
>         }
>
> -       qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> -
>         /*
> -        * Let's got through all collected functions and filter
> -        * out those that are not in ftrace.
> +        * Sort both functions and addrs so we can iterate
> +        * both of them simultaneously and found matching
> +        * func/addr pairs.
>          */
> -       for (i = 0; i < functions_cnt; i++) {
> -               struct elf_function *func = &functions[i];
> -               /*
> -                * For vmlinux image both addrs[x] and functions[x]::addr
> -                * values are final address and are comparable.
> -                *
> -                * For kernel module addrs[x] is final address, but
> -                * functions[x]::addr is relative address within section
> -                * and needs to be relocated by adding sh_addr.
> -                */
> -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> +       qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
> +
> +       for (i_func = 0, i_addr = 0; i_func < functions_cnt; i_func++) {
> +               struct elf_function *func = &functions[i_func];
> +
> +               if (i_func + 1 < functions_cnt)
> +                       func->end = functions[i_func + 1].addr;
> +
> +               for (; i_addr < count; i_addr++) {
> +                       __u64 addr = addrs[i_addr];
> +
> +                       /* Functions are  ahead, catch up with addrs. */
> +                       if (addr < func->addr)
> +                               continue;
> +
> +                       /* Addr is within function - mark function as valid. */
> +                       if (is_addr_in_func(addr, func, kmod)) {
> +                               /*
> +                                * We iterate over sorted array, so we can easily skip
> +                                * not valid item and move following valid field into
> +                                * its place, and still keep the 'new' array sorted.
> +                                */
> +                               if (i_func != functions_valid)
> +                                       functions[functions_valid] = functions[i_func];
> +                               functions_valid++;
> +                               i_addr++;
> +                       }
>
> -               /* Make sure function is within ftrace addresses. */
> -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
>                         /*
> -                        * We iterate over sorted array, so we can easily skip
> -                        * not valid item and move following valid field into
> -                        * its place, and still keep the 'new' array sorted.
> +                        * Addrs are ahead, catch up with functions, or we just
> +                        * found valid function and want to move to another.
>                          */
> -                       if (i != functions_valid)
> -                               functions[functions_valid] = functions[i];
> -                       functions_valid++;
> +                       break;
>                 }
>         }
>
> +       if (btf_elf__verbose) {
> +               printf("Found %d functions out of %d symbols and %llu ftrace addresses.\n",
> +                       functions_valid, functions_cnt, count);
> +       }
> +
>         functions_cnt = functions_valid;
>         free(addrs);
>
> -       if (btf_elf__verbose)
> -               printf("Found %d functions!\n", functions_cnt);
> +       /*
> +        * And finaly sort 'valid' functions by name,
> +        * so find_function can be used.
> +        */
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
> +
>         return 0;
>  }
>
> @@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
>         struct elf_function key = { .name = name };
>
>         return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> -                      functions_cmp);
> +                      functions_cmp_name);
>  }
>
>  static bool btf_name_char_ok(char c, bool first)
>

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Sedat Dilek]

On Wed, Feb 10, 2021 at 7:20 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Wed, Feb 10, 2021 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Feb 09, 2021 at 02:00:29PM -0800, Andrii Nakryiko wrote:
> >
> > SNIP
> >
> > > > > > I'm still trying to build the kernel.. however ;-)
> > > > > >
> > > > > > patch below adds the ftrace check only for static functions
> > > > > > and lets the externa go through.. but as you said, in this
> > > > > > case we'll need to figure out the 'notrace' and other checks
> > > > > > ftrace is doing
> > > > > >
> > > > > > jirka
> > > > > >
> > > > > >
> > > > > > ---
> > > > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > > > index b124ec20a689..4d147406cfa5 100644
> > > > > > --- a/btf_encoder.c
> > > > > > +++ b/btf_encoder.c
> > > > > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > > > > >                         continue;
> > > > > >                 if (!has_arg_names(cu, &fn->proto))
> > > > > >                         continue;
> > > > > > -               if (functions_cnt) {
> > > > > > +               if (!fn->external && functions_cnt) {
> > > > >
> > > > > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > > > > FUNC ELF symbol be more reliable?
> > > >
> > > > that'd mean extra bsearch on each processed function,
> > > > on the ther hand, we'are already slow ;-) I'll check
> > > > how big the slowdown would be
> > > >
> > >
> > > We currently record addresses and do binary search. Now we need to
> > > record address + size and still do binary search with a slightly
> > > different semantics (find closest entry >= addr). Then just check that
> > > it overlaps, taking into account the length of the function code. It
> > > shouldn't result in a noticeable slowdown. Might be actually faster,
> > > because we might avoid callback function call costs.
> >
> > I'm still not sure how to handle the external check for function via elf,
>
> I might be missing something, but don't all functions have
> corresponding ELF symbols? And then symbol can have LOCAL or GLOBAL
> type. LOCALs are supposed to be not visible outside respective CUs (so
> correspond to static functions), while GLOBALs are extern-able funcs.
> So if func's symbol is GLOBAL, it should be ok to assume it's
> attachable (not inlined, at least).
>
> > but below is change for checking that ftrace addrs are within elf functions
> >
> > seems to work in my tests, I'll run some more tests and send full patch
>
> It seems unnecessarily convoluted. I was thinking about something like
> this (the diff will totally be screwed up by gmail, and I haven't even
> compiled it):
>

Now, I know why vfs_truncate is so problematic.
Let's split it: "vfs + trunc + ate".
Eaten by Gmail :-).

- Sedat -


> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..8162b238bd43 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
> **paddrs, __u64 *pcount)
>         return 0;
>  }
>
> +struct func_seg { __u64 start; __u64 end; };
> +
> +static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
> +{
> +       size_t l = 0, r = len - 1, m;
> +
> +       while (l < r) {
> +               m = l + (r - l + 1) / 2;
> +               if (segs[m].start <= addr)
> +                       l = m;
> +               else
> +                       r = m - 1;
> +       }
> +
> +       return segs[l].start <= addr && addr < segs[l].end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
>         __u64 *addrs, count, i;
> @@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
> struct funcs_layout *fl)
>                 __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
>
>                 /* Make sure function is within ftrace addresses. */
> -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> +               if (func_exists(addrs, count, addr))
>                         /*
>                          * We iterate over sorted array, so we can easily skip
>                          * not valid item and move following valid field into
>
>
> So the idea is to use address segments and check whether there is a
> segment that overlaps with a given address by first binary searching
> for a segment with the largest starting address that is <= addr. And
> then just confirming that segment does overlap with the requested
> address.
>
> WDYT?
>
> >
> > jirka
> >
> >
> > ---
> > diff --git a/btf_encoder.c b/btf_encoder.c
> > index b124ec20a689..548a12847f99 100644
> > --- a/btf_encoder.c
> > +++ b/btf_encoder.c
> > @@ -36,6 +36,7 @@ struct funcs_layout {
> >  struct elf_function {
> >         const char      *name;
> >         unsigned long    addr;
> > +       unsigned long    end;
> >         unsigned long    sh_addr;
> >         bool             generated;
> >  };
> > @@ -44,7 +45,7 @@ static struct elf_function *functions;
> >  static int functions_alloc;
> >  static int functions_cnt;
> >
> > -static int functions_cmp(const void *_a, const void *_b)
> > +static int functions_cmp_name(const void *_a, const void *_b)
> >  {
> >         const struct elf_function *a = _a;
> >         const struct elf_function *b = _b;
> > @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
> >         return strcmp(a->name, b->name);
> >  }
> >
> > +static int functions_cmp_addr(const void *_a, const void *_b)
> > +{
> > +       const struct elf_function *a = _a;
> > +       const struct elf_function *b = _b;
> > +
> > +       if (a->addr == b->addr)
> > +               return 0;
> > +       return a->addr < b->addr ? -1 : 1;
> > +}
> > +
> >  static void delete_functions(void)
> >  {
> >         free(functions);
> > @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
> >
> >         functions[functions_cnt].name = name;
> >         functions[functions_cnt].addr = elf_sym__value(sym);
> > +       functions[functions_cnt].end = (__u64) -1;
> >         functions[functions_cnt].sh_addr = sh.sh_addr;
> >         functions[functions_cnt].generated = false;
> >         functions_cnt++;
> > @@ -236,9 +248,25 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
> >         return 0;
> >  }
> >
> > +static bool is_addr_in_func(__u64 addr, struct elf_function *func, bool kmod)
> > +{
> > +       /*
> > +        * For vmlinux image both addrs[x] and functions[x]::addr
> > +        * values are final address and are comparable.
> > +        *
> > +        * For kernel module addrs[x] is final address, but
> > +        * functions[x]::addr is relative address within section
> > +        * and needs to be relocated by adding sh_addr.
> > +        */
> > +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> > +       __u64 end = kmod ? func->end+ func->sh_addr : func->end;
> > +
> > +       return start <= addr && addr < end;
> > +}
> > +
> >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >  {
> > -       __u64 *addrs, count, i;
> > +       __u64 *addrs, count, i_func, i_addr;
> >         int functions_valid = 0;
> >         bool kmod = false;
> >
> > @@ -266,43 +294,62 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >                 return 0;
> >         }
> >
> > -       qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> > -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> > -
> >         /*
> > -        * Let's got through all collected functions and filter
> > -        * out those that are not in ftrace.
> > +        * Sort both functions and addrs so we can iterate
> > +        * both of them simultaneously and found matching
> > +        * func/addr pairs.
> >          */
> > -       for (i = 0; i < functions_cnt; i++) {
> > -               struct elf_function *func = &functions[i];
> > -               /*
> > -                * For vmlinux image both addrs[x] and functions[x]::addr
> > -                * values are final address and are comparable.
> > -                *
> > -                * For kernel module addrs[x] is final address, but
> > -                * functions[x]::addr is relative address within section
> > -                * and needs to be relocated by adding sh_addr.
> > -                */
> > -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > +       qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> > +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
> > +
> > +       for (i_func = 0, i_addr = 0; i_func < functions_cnt; i_func++) {
> > +               struct elf_function *func = &functions[i_func];
> > +
> > +               if (i_func + 1 < functions_cnt)
> > +                       func->end = functions[i_func + 1].addr;
> > +
> > +               for (; i_addr < count; i_addr++) {
> > +                       __u64 addr = addrs[i_addr];
> > +
> > +                       /* Functions are  ahead, catch up with addrs. */
> > +                       if (addr < func->addr)
> > +                               continue;
> > +
> > +                       /* Addr is within function - mark function as valid. */
> > +                       if (is_addr_in_func(addr, func, kmod)) {
> > +                               /*
> > +                                * We iterate over sorted array, so we can easily skip
> > +                                * not valid item and move following valid field into
> > +                                * its place, and still keep the 'new' array sorted.
> > +                                */
> > +                               if (i_func != functions_valid)
> > +                                       functions[functions_valid] = functions[i_func];
> > +                               functions_valid++;
> > +                               i_addr++;
> > +                       }
> >
> > -               /* Make sure function is within ftrace addresses. */
> > -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> >                         /*
> > -                        * We iterate over sorted array, so we can easily skip
> > -                        * not valid item and move following valid field into
> > -                        * its place, and still keep the 'new' array sorted.
> > +                        * Addrs are ahead, catch up with functions, or we just
> > +                        * found valid function and want to move to another.
> >                          */
> > -                       if (i != functions_valid)
> > -                               functions[functions_valid] = functions[i];
> > -                       functions_valid++;
> > +                       break;
> >                 }
> >         }
> >
> > +       if (btf_elf__verbose) {
> > +               printf("Found %d functions out of %d symbols and %llu ftrace addresses.\n",
> > +                       functions_valid, functions_cnt, count);
> > +       }
> > +
> >         functions_cnt = functions_valid;
> >         free(addrs);
> >
> > -       if (btf_elf__verbose)
> > -               printf("Found %d functions!\n", functions_cnt);
> > +       /*
> > +        * And finaly sort 'valid' functions by name,
> > +        * so find_function can be used.
> > +        */
> > +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
> > +
> >         return 0;
> >  }
> >
> > @@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
> >         struct elf_function key = { .name = name };
> >
> >         return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> > -                      functions_cmp);
> > +                      functions_cmp_name);
> >  }
> >
> >  static bool btf_name_char_ok(char c, bool first)
> >
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAEf4BzbzquqsA5%3D_UqDukScuoGLfDhZiiXs_sgYBuNUvTBuV6w%40mail.gmail.com.

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:
> On Wed, Feb 10, 2021 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Tue, Feb 09, 2021 at 02:00:29PM -0800, Andrii Nakryiko wrote:
> >
> > SNIP
> >
> > > > > > I'm still trying to build the kernel.. however ;-)
> > > > > >
> > > > > > patch below adds the ftrace check only for static functions
> > > > > > and lets the externa go through.. but as you said, in this
> > > > > > case we'll need to figure out the 'notrace' and other checks
> > > > > > ftrace is doing
> > > > > >
> > > > > > jirka
> > > > > >
> > > > > >
> > > > > > ---
> > > > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > > > index b124ec20a689..4d147406cfa5 100644
> > > > > > --- a/btf_encoder.c
> > > > > > +++ b/btf_encoder.c
> > > > > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > > > > >                         continue;
> > > > > >                 if (!has_arg_names(cu, &fn->proto))
> > > > > >                         continue;
> > > > > > -               if (functions_cnt) {
> > > > > > +               if (!fn->external && functions_cnt) {
> > > > >
> > > > > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > > > > FUNC ELF symbol be more reliable?
> > > >
> > > > that'd mean extra bsearch on each processed function,
> > > > on the ther hand, we'are already slow ;-) I'll check
> > > > how big the slowdown would be
> > > >
> > >
> > > We currently record addresses and do binary search. Now we need to
> > > record address + size and still do binary search with a slightly
> > > different semantics (find closest entry >= addr). Then just check that
> > > it overlaps, taking into account the length of the function code. It
> > > shouldn't result in a noticeable slowdown. Might be actually faster,
> > > because we might avoid callback function call costs.
> >
> > I'm still not sure how to handle the external check for function via elf,
> 
> I might be missing something, but don't all functions have
> corresponding ELF symbols? And then symbol can have LOCAL or GLOBAL
> type. LOCALs are supposed to be not visible outside respective CUs (so
> correspond to static functions), while GLOBALs are extern-able funcs.
> So if func's symbol is GLOBAL, it should be ok to assume it's
> attachable (not inlined, at least).

sure I know where the info is, I was just hesitating to add another bsearch

anyway, I checked the BTF data before and after this change (the dwarf check above)
and it looks like there are many global functions that are not attachable

attaching diff of BTF functions below - before (.old) and after (.new) adding
all external functions.. all added (+) functions are external and do not have
ftrace address - I verified some of them in disassembly and they are not attachable

I think we should check ftrace addresses also for global functions 

I'm of course assuming BTF should have only attachable functions

jirka


---
--- /tmp/pahole.test.PbD/funcs-vmlinux-x86-gcc-1.old	2021-02-10 19:51:44.727865204 +0100
+++ /tmp/pahole.test.PbD/funcs-vmlinux-x86-gcc-1.new	2021-02-10 19:51:44.836865827 +0100
@@ -1662,6 +1662,7 @@
 'add_to_swap'
 'add_to_swap_cache'
 'add_transaction_credits'
+'add_uevent_var'
 'add_user_to_lr'
 'add_wait_queue'
 'add_wait_queue_exclusive'
@@ -1989,12 +1990,14 @@
 'allocate_cgrp_cset_links'
 'allocate_fake_cpuc'
 'allocate_file_region_entries'
+'allocate_ftrace_func_mapper'
 'allocate_resource'
 'allocate_slab'
 'allocate_threshold_blocks'
 'allocation_policy_show'
 '__alloc_bootmem_huge_page'
 'alloc_bprm'
+'__alloc_bucket_spinlocks'
 'alloc_buffer'
 'alloc_buffer_head'
 'alloc_calls_show'
@@ -2003,6 +2006,7 @@
 'alloc_cipher_test_sglists'
 'alloc_contig_pages'
 'alloc_contig_range'
+'alloc_cpu_rmap'
 'alloc_dax'
 'alloc_dax_region'
 'alloc_debug_processing'
@@ -2048,6 +2052,7 @@
 'alloc_page_buffers'
 '__alloc_pages_direct_compact'
 'alloc_pages_exact'
+'alloc_pages_exact_nid'
 '__alloc_pages_nodemask'
 'alloc_pci_root_info'
 '__alloc_percpu'
@@ -2299,6 +2304,7 @@
 'apple_airport_reset'
 'apply_alternatives'
 'apply_constraint'
+'apply_event_filter'
 '__apply_microcode_amd'
 'apply_microcode_amd'
 'apply_microcode_early'
@@ -2307,6 +2313,7 @@
 'apply_mlockall_flags'
 'apply_paravirt'
 'apply_relocate_add'
+'apply_subsystem_event_filter'
 'apply_to_existing_page_range'
 '__apply_to_page_range'
 'apply_to_page_range'
@@ -2345,6 +2352,11 @@
 'arch_early_irq_init'
 'arch_freq_get_on_cpu'
 'arch_freq_prepare_all'
+'arch_ftrace_match_adjust'
+'arch_ftrace_trampoline_free'
+'arch_ftrace_trampoline_func'
+'arch_ftrace_update_code'
+'arch_ftrace_update_trampoline'
 'arch_get_kallsym'
 'arch_get_platform_mac_address'
 'arch_get_unmapped_area'
@@ -2420,17 +2432,20 @@
 'arch_setup_new_exec'
 'arch_set_user_pkey_access'
 'arch_show_interrupts'
+'arch_smt_update'
 'arch_stack_walk'
 'arch_stack_walk_reliable'
 'arch_stack_walk_user'
 'arch_static_call_transform'
 'arch_suspend_disable_irqs'
 'arch_suspend_enable_irqs'
+'arch_syscall_addr'
 'arch_syscall_is_vdso_sigreturn'
 'arch_task_cache_init'
 'arch_thaw_secondary_cpus_begin'
 'arch_thaw_secondary_cpus_end'
 'arch_tlbbatch_flush'
+'arch_touch_nmi_watchdog'
 'arch_trampoline_kprobe'
 'arch_trigger_cpumask_backtrace'
 'arch_uninstall_hw_breakpoint'
@@ -2454,6 +2469,8 @@
 'arch_wb_cache_pmem'
 'arch_within_kprobe_blacklist'
 'arch_within_optimized_kprobe'
+'argv_free'
+'argv_split'
 'ari_enabled_show'
 'arm_kprobe'
 'arm_timer'
@@ -2504,6 +2521,7 @@
 'array_size_store'
 'array_state_show'
 'array_state_store'
+'asn1_ber_decoder'
 'aspm_ctrl_attrs_are_visible'
 'assert_show'
 'assign_fw'
@@ -2511,6 +2529,16 @@
 'assign_requested_resources_sorted'
 '__assign_resources_sorted'
 'assign_vector_locked'
+'assoc_array_apply_edit'
+'assoc_array_cancel_edit'
+'assoc_array_clear'
+'assoc_array_delete'
+'assoc_array_destroy'
+'assoc_array_find'
+'assoc_array_gc'
+'assoc_array_insert'
+'assoc_array_insert_set_object'
+'assoc_array_iterate'
 'asus_hides_ac97_lpc'
 'asus_hides_smbus_hostbridge'
 'asus_hides_smbus_lpc'
@@ -2525,6 +2553,7 @@
 'asymmetric_key_eds_op'
 'asymmetric_key_free_preparse'
 'asymmetric_key_generate_id'
+'__asymmetric_key_hex_to_key_id'
 'asymmetric_key_hex_to_key_id'
 'asymmetric_key_id_partial'
 'asymmetric_key_id_same'
@@ -2955,6 +2984,8 @@
 'atom_get_turbo_pstate'
 'atom_get_val'
 'atom_get_vid'
+'_atomic_dec_and_lock'
+'_atomic_dec_and_lock_irqsave'
 'atomic_dec_and_mutex_lock'
 'atomic_notifier_call_chain'
 'atomic_notifier_call_chain_robust'
@@ -3230,6 +3261,7 @@
 'avc_has_extended_perms'
 'avc_has_perm'
 'avc_has_perm_flags'
+'avc_has_perm_noaudit'
 'avc_init'
 'avc_lookup'
 'avc_node_free'
@@ -3343,11 +3375,13 @@
 'battery_notify'
 'bb_show'
 'bb_store'
+'_bcd2bin'
 'bcdDevice_show'
 'bc_handler'
 'bcj_apply'
 'bcj_flush'
 'bclean'
+'bcmp'
 'bConfigurationValue_show'
 'bConfigurationValue_store'
 'bc_set_next'
@@ -3585,6 +3619,8 @@
 'bictcp_state'
 'bic_to_bfqd'
 'bic_to_bfqq'
+'_bin2bcd'
+'bin2hex'
 'bin_attr_nvmem_read'
 'bin_attr_nvmem_write'
 '__bind'
@@ -3668,7 +3704,37 @@
 'bitfill_aligned_rev'
 'bitfill_unaligned'
 'bitfill_unaligned_rev'
+'bitmap_alloc'
+'bitmap_allocate_region'
+'__bitmap_and'
+'__bitmap_andnot'
+'__bitmap_clear'
+'__bitmap_complement'
+'bitmap_cut'
+'__bitmap_equal'
+'bitmap_find_free_region'
+'bitmap_find_next_zero_area_off'
+'bitmap_free'
+'bitmap_from_arr32'
+'__bitmap_intersects'
+'__bitmap_or'
+'__bitmap_or_equal'
+'bitmap_parse'
+'bitmap_parselist'
+'bitmap_parselist_user'
+'bitmap_parse_user'
+'bitmap_print_to_pagebuf'
+'bitmap_release_region'
+'__bitmap_replace'
+'__bitmap_set'
+'__bitmap_shift_left'
+'__bitmap_shift_right'
 'bitmap_store'
+'__bitmap_subset'
+'bitmap_to_arr32'
+'__bitmap_weight'
+'__bitmap_xor'
+'bitmap_zalloc'
 'bit_putcs'
 'bit_update_start'
 'bit_wait'
@@ -3785,6 +3851,7 @@
 'blk_end_sync_rq'
 'blk_execute_rq'
 'blk_execute_rq_nowait'
+'blk_fill_rwbs'
 'blk_finish_plug'
 'blk_flush_complete_seq'
 'blk_flush_integrity'
@@ -3880,6 +3947,7 @@
 'blk_mq_dispatch_wake'
 'blk_mq_do_dispatch_ctx'
 '__blk_mq_do_dispatch_sched'
+'__blk_mq_end_request'
 'blk_mq_end_request'
 'blk_mq_exit_hctx'
 'blk_mq_exit_queue'
@@ -4005,6 +4073,7 @@
 'blk_mq_virtio_map_queues'
 'blk_mq_wake_waiters'
 'blk_next_bio'
+'blk_op_str'
 'blkpg_do_ioctl'
 'blk_pm_runtime_init'
 'blk_poll'
@@ -4202,15 +4271,18 @@
 'bpf_csum_level'
 'bpf_csum_update'
 'bpf_ctx_init'
+'bpf_current_task_under_cgroup'
 'bpf_destroy_state'
 'bpf_dispatcher_change_prog'
 'bpf_dispatcher_prepare'
 'bpf_dispatcher_xdp_func'
+'bpf_d_path'
 'bpf_dummy_read'
 'bpf_dummy_write'
 'bpf_encap_cmp'
 'bpf_encap_nlsize'
 '__bpf_event_entry_free'
+'bpf_event_output'
 'bpf_event_output_data'
 'bpf_exit_net'
 'bpf_fd_array_map_lookup_elem'
@@ -4254,18 +4326,23 @@
 'bpf_get_current_cgroup_id'
 'bpf_get_current_comm'
 'bpf_get_current_pid_tgid'
+'bpf_get_current_task'
+'bpf_get_current_task_btf'
 'bpf_get_current_uid_gid'
 'bpf_get_file_flag'
 'bpf_get_hash_recalc'
 'bpf_get_inode'
 'bpf_get_kallsym'
+'bpf_get_kprobe_info'
 'bpf_get_listener_sock'
 'bpf_get_local_storage'
 'bpf_get_netns_cookie_sock'
 'bpf_get_netns_cookie_sock_addr'
 'bpf_get_ns_current_pid_tgid'
 'bpf_get_numa_node_id'
+'bpf_get_perf_event_info'
 'bpf_get_raw_cpu_id'
+'bpf_get_raw_tracepoint'
 'bpf_get_route_realm'
 'bpf_get_skb_set_tunnel_proto'
 'bpf_get_smp_processor_id'
@@ -4280,9 +4357,15 @@
 '__bpf_get_stackid'
 'bpf_get_stackid'
 'bpf_get_stackid_pe'
+'bpf_get_stackid_raw_tp'
+'bpf_get_stackid_tp'
 'bpf_get_stack_pe'
+'bpf_get_stack_raw_tp'
+'bpf_get_stack_tp'
 'bpf_get_task_stack'
+'bpf_get_trace_printk_proto'
 'bpf_get_tree'
+'bpf_get_uprobe_info'
 'bpf_hash_map_seq_find_next'
 'bpf_hash_map_seq_next'
 '__bpf_hash_map_seq_show'
@@ -4741,9 +4824,24 @@
 'bpf_percpu_hash_copy'
 'bpf_percpu_hash_update'
 'bpf_per_cpu_ptr'
+'bpf_perf_event_output'
+'bpf_perf_event_output_raw_tp'
+'bpf_perf_event_output_tp'
+'bpf_perf_event_read'
+'bpf_perf_event_read_value'
+'bpf_perf_prog_read_value'
 'bpf_pid_task_storage_delete_elem'
 'bpf_pid_task_storage_lookup_elem'
 'bpf_pid_task_storage_update_elem'
+'bpf_probe_read_compat'
+'bpf_probe_read_compat_str'
+'bpf_probe_read_kernel'
+'bpf_probe_read_kernel_str'
+'bpf_probe_read_user'
+'bpf_probe_read_user_str'
+'bpf_probe_register'
+'bpf_probe_unregister'
+'bpf_probe_write_user'
 'bpf_prog_add'
 'bpf_prog_alloc'
 'bpf_prog_alloc_jited_linfo'
@@ -4766,6 +4864,9 @@
 'bpf_prog_create'
 'bpf_prog_create_from_user'
 'bpf_prog_destroy'
+'__bpf_prog_enter'
+'__bpf_prog_enter_sleepable'
+'__bpf_prog_exit'
 '__bpf_prog_exit_sleepable'
 'bpf_prog_fill_jited_linfo'
 '__bpf_prog_free'
@@ -4857,6 +4958,7 @@
 'bpf_prog_test_run_xdp'
 'bpf_prog_warn_on_exec'
 'bpf_push_seg6_encap'
+'bpf_put_raw_tracepoint'
 'bpf_raw_tp_link_dealloc'
 'bpf_raw_tp_link_fill_link_info'
 'bpf_raw_tp_link_release'
@@ -4865,6 +4967,7 @@
 'bpf_rc_keydown'
 'bpf_rc_pointer_rel'
 'bpf_rc_repeat'
+'bpf_read_branch_records'
 '__bpf_redirect'
 'bpf_redirect'
 'bpf_redirect_neigh'
@@ -4886,7 +4989,12 @@
 'bpf_selem_unlink_map'
 '__bpf_selem_unlink_storage'
 'bpf_selem_unlink_storage_nolock'
+'bpf_send_signal'
+'bpf_send_signal_thread'
+'bpf_seq_printf'
+'bpf_seq_printf_btf'
 'bpf_seq_read'
+'bpf_seq_write'
 'bpf_set_hash'
 'bpf_set_hash_invalid'
 '_bpf_setsockopt'
@@ -4969,6 +5077,7 @@
 'bpf_sk_storage_ptr'
 'bpf_sk_storage_tracing_allowed'
 'bpf_sk_storage_uncharge'
+'bpf_snprintf_btf'
 'bpf_sock_addr_getsockopt'
 'bpf_sock_addr_setsockopt'
 'bpf_sock_addr_skc_lookup_tcp'
@@ -5034,6 +5143,20 @@
 'bpf_tcp_sock_is_valid_access'
 'bpf_test_run'
 'bpf_this_cpu_ptr'
+'bpf_trace_printk'
+'bpf_trace_run1'
+'bpf_trace_run10'
+'bpf_trace_run11'
+'bpf_trace_run12'
+'bpf_trace_run2'
+'bpf_trace_run3'
+'bpf_trace_run4'
+'bpf_trace_run5'
+'bpf_trace_run6'
+'bpf_trace_run7'
+'bpf_trace_run8'
+'bpf_trace_run9'
+'bpf_tracing_func_proto'
 'bpf_tracing_link_dealloc'
 'bpf_tracing_link_fill_link_info'
 'bpf_tracing_link_release'
@@ -5072,6 +5195,7 @@
 'bpf_xdp_sock_is_valid_access'
 'bpf_xmit'
 'bp_perf_event_destroy'
+'bprintf'
 'bprm_change_interp'
 'bprm_execve'
 'bq_enqueue'
@@ -5116,6 +5240,7 @@
 'broken_parity_status_show'
 'broken_parity_status_store'
 'broken_suspend'
+'bsearch'
 'bsg_complete'
 'bsg_devnode'
 'bsg_exit_rq'
@@ -5150,6 +5275,7 @@
 'bsp_init_intel'
 'bsp_pm_callback'
 'bsp_pm_check_init'
+'bstr_printf'
 'bsw_pwm_setup'
 'btf_alloc_id'
 'btf_array_check_member'
@@ -5292,6 +5418,9 @@
 'build_all_zonelists_init'
 'build_attr'
 'build_cipher_test_sglists'
+'build_id_parse'
+'build_open_flags'
+'build_open_how'
 'build_sched_domains'
 '__build_skb'
 'build_skb'
@@ -5301,6 +5430,7 @@
 'build_tree'
 'build_zonelists'
 'bump_cpu_timer'
+'bunzip2'
 'bus_add_device'
 'bus_add_driver'
 'bus_attr_show'
@@ -5327,6 +5457,7 @@
 'bus_rescan_store'
 'bus_set_iommu'
 'bus_sort_breadthfirst'
+'bust_spinlocks'
 'bus_uevent_filter'
 'bus_uevent_store'
 'bus_unregister'
@@ -5449,6 +5580,7 @@
 'calculate_totalreserve_pages'
 'calc_wheel_index'
 'calibrate_delay'
+'calibrate_delay_is_known'
 'calibration_delay_done'
 'calipso_cache_add'
 'calipso_cache_entry_free'
@@ -5492,6 +5624,7 @@
 'call_fib_notifier'
 'call_fib_notifiers'
 'call_filldir'
+'call_filter_check_discard'
 'call_function_init'
 'call_netdevice_notifiers'
 'call_netdevice_notifiers_info'
@@ -6021,6 +6154,7 @@
 'checkreqprot_setup'
 'check_root_hub_suspended'
 '__check_shared_memory'
+'check_signature'
 'check_single_step'
 'check_slab'
 'check_slow_task'
@@ -6032,8 +6166,10 @@
 'check_syslog_permissions'
 'check_tsc_sync_source'
 'check_tsc_sync_target'
+'check_tsc_unstable'
 'check_tsc_warp'
 'check_via_agp3'
+'check_zeroed_user'
 'check_zero_holes'
 'children_seq_next'
 'children_seq_open'
@@ -6211,6 +6347,8 @@
 'clear_buddies'
 'clear_buffer_attributes'
 'clear_cpu_cap'
+'clear_event_triggers'
+'clear_ftrace_function_probes'
 'clear_huge_page'
 'clear_inode'
 'clear_IO_APIC'
@@ -6227,6 +6365,7 @@
 'clear_refs_pte_range'
 'clear_refs_test_walk'
 'clear_refs_write'
+'clear_sched_clock_stable'
 'clear_selection'
 'clear_shadow_from_swap_cache'
 'clear_show'
@@ -6564,6 +6703,8 @@
 'cls_fl_init'
 'cls_flow_init'
 'cls_mall_init'
+'__clzdi2'
+'__clzsi2'
 'cma_alloc'
 'cma_clear_bitmap'
 'cma_declare_contiguous_nid'
@@ -6696,6 +6837,7 @@
 'common_hrtimer_remaining'
 'common_hrtimer_try_to_cancel'
 'common_index'
+'common_interrupt'
 'common_lsm_audit'
 'common_nsleep'
 'common_nsleep_timens'
@@ -6936,6 +7078,8 @@
 'con_unify_unimap'
 'con_unthrottle'
 'conv_8bit_to_uni'
+'convert_art_ns_to_tsc'
+'convert_art_to_tsc'
 'convert_bpf_ld_abs'
 'convert_context'
 'convert_ctx_accesses'
@@ -6970,10 +7114,16 @@
 'copy_fpstate_to_sigframe'
 'copy_from_buf'
 'copy_from_early_mem'
+'_copy_from_iter'
+'_copy_from_iter_flushcache'
+'_copy_from_iter_full'
+'_copy_from_iter_full_nocache'
+'_copy_from_iter_nocache'
 'copy_from_kernel_nofault'
 'copy_from_kernel_nofault_allowed'
 'copy_from_page'
 'copy_from_read_buf'
+'_copy_from_user'
 'copy_from_user_nmi'
 'copy_from_user_nofault'
 'copy_from_user_policy'
@@ -6991,6 +7141,8 @@
 'copy_kernel_to_dynamic_supervisor'
 'copy_kernel_to_xstate'
 'copy_map_value_locked'
+'copy_mc_fragile_handle_tail'
+'_copy_mc_to_iter'
 'copy_mc_to_kernel'
 'copy_mc_to_user'
 'copy_mnt_ns'
@@ -7001,7 +7153,9 @@
 'copy_oldmem_page'
 'copy_oldmem_page_encrypted'
 'copy_optimized_instructions'
+'copy_page_from_iter'
 'copy_page_range'
+'copy_page_to_iter'
 'copy_part'
 'copy_pid_ns'
 'copy_process'
@@ -7023,7 +7177,11 @@
 'copy_thread'
 'copy_time_ns'
 'copy_to_high_bio_irq'
+'_copy_to_iter'
+'copy_to_kernel_nofault'
 'copy_to_page'
+'_copy_to_user'
+'copy_to_user_nofault'
 'copy_to_user_policy'
 'copy_to_user_state'
 'copy_to_user_state_extra'
@@ -7058,6 +7216,7 @@
 'core_in_sync'
 'core_is_clean'
 'core_kernel_data'
+'core_kernel_text'
 'core_mark_region'
 'core_pmu_enable_all'
 'core_pmu_enable_event'
@@ -7147,6 +7306,8 @@
 'cpu_clock_sample'
 'cpu_clock_sample_group'
 'cpu_coregroup_mask'
+'cpu_detect'
+'cpu_detect_cache_sizes'
 'cpu_detect_tlb_amd'
 'cpu_detect_tlb_hygon'
 'cpu_device_create'
@@ -7348,6 +7509,8 @@
 'cpuid_smp_cpuid'
 'cpuinfo_open'
 'cpu_in_idle'
+'cpu_init'
+'cpu_init_exception_handling'
 'cpu_init_udelay'
 'cpu_is_hotpluggable'
 'cpu_khz_from_msr'
@@ -7379,6 +7542,13 @@
 'cpu_maps_update_begin'
 'cpu_maps_update_done'
 'cpu_map_update_elem'
+'cpumask_any_and_distribute'
+'cpumask_any_but'
+'cpumask_any_distribute'
+'cpumask_local_spread'
+'cpumask_next'
+'cpumask_next_and'
+'cpumask_next_wrap'
 'cpu_max_show'
 'cpu_max_write'
 'cpu_mitigations_auto_nosmt'
@@ -7392,6 +7562,9 @@
 'cpupri_set'
 'cpu_report_death'
 'cpu_report_state'
+'cpu_rmap_add'
+'cpu_rmap_put'
+'cpu_rmap_update'
 'cpuset_attach'
 'cpuset_bind'
 'cpuset_can_attach'
@@ -7487,6 +7660,9 @@
 'cpu_weight_write_u64'
 'cqm_handle_limbo'
 'cqm_setup_limbo_handler'
+'cr4_init'
+'cr4_read_shadow'
+'cr4_update_irqsoff'
 'cr4_update_pce'
 'crash_free_reserved_phys_range'
 'crash_get_memory_size'
@@ -7518,22 +7694,37 @@
 'crb_request_locality'
 'crb_send'
 'crb_status'
+'crc16'
+'crc32_be'
+'crc32c'
 'crc32c_cra_init'
+'crc32c_impl'
+'__crc32c_le'
+'__crc32c_le_shift'
 'crc32c_mod_init'
+'crc32_le'
+'crc32_le_shift'
 'crc32_threadfn'
+'crc_ccitt'
+'crc_ccitt_false'
 'crc_pwm_apply'
 'crc_pwm_get_state'
+'crc_t10dif'
 'crc_t10dif_generic'
 'crct10dif_mod_init'
+'crc_t10dif_update'
 'create_basic_memory_bitmaps'
 'create_boot_cache'
 'create_default_group'
 '__create_dir'
 'create_empty_buffers'
+'create_event_filter'
 'create_init_pkru_value'
 'create_io_worker'
 'create_kmalloc_cache'
 'create_kmalloc_caches'
+'create_local_trace_kprobe'
+'create_local_trace_uprobe'
 'create_log_context'
 'create_new_namespaces'
 'create_of_modalias'
@@ -7717,6 +7908,7 @@
 'crypto_ecb_module_init'
 'crypto_enqueue_request'
 'crypto_enqueue_request_head'
+'crypto_exit_proc'
 'crypto_exit_scomp_ops_async'
 'crypto_exit_shash_ops_async'
 'crypto_find_alg'
@@ -7947,7 +8139,10 @@
 'css_tryget_online_from_dir'
 'c_start'
 'c_stop'
+'csum_and_copy_from_iter'
+'csum_and_copy_from_iter_full'
 'csum_and_copy_from_user'
+'csum_and_copy_to_iter'
 'csum_and_copy_to_user'
 'csum_exit_net'
 'csum_init_module'
@@ -7984,6 +8179,8 @@
 'ctx_read_rq_list_next'
 'ctx_read_rq_list_start'
 'ctx_read_rq_list_stop'
+'__ctzdi2'
+'__ctzsi2'
 'cubictcp_register'
 'cur_freq_show'
 'current_chrooted'
@@ -7992,6 +8189,7 @@
 'current_cpuset_is_being_rebound'
 'current_in_userns'
 'current_is_async'
+'current_is_single_threaded'
 'current_is_workqueue_rescuer'
 'current_link_speed_show'
 'current_link_width_show'
@@ -8028,6 +8226,8 @@
 'cwt_wakefn'
 'cw_update_attr'
 'cw_update_start'
+'cyc2ns_read_begin'
+'cyc2ns_read_end'
 'cyc_show'
 'cyc_thresh_show'
 'cypress_detect'
@@ -8061,6 +8261,7 @@
 'date_show'
 'dax_alive'
 'dax_alloc_inode'
+'dax_bus_exit'
 'dax_bus_init'
 'dax_bus_match'
 'dax_bus_probe'
@@ -8211,6 +8412,9 @@
 'dcookie_unregister'
 'dd_bio_merge'
 'dd_dispatch_request'
+'ddebug_add_module'
+'ddebug_dyndbg_module_param_cb'
+'ddebug_remove_module'
 'd_delete'
 'dd_exit_queue'
 'dd_finish_request'
@@ -8330,12 +8534,14 @@
 'debugfs_ulong_set'
 'debugfs_write_file_bool'
 'debug_kernel'
+'debug_locks_off'
 'debug_mount'
 'debug_output'
 'debug_periodic_open'
 'debug_prepare_data'
 'debug_registers_open'
 'debug_reply_size'
+'debug_smp_processor_id'
 'decap_and_validate'
 'dec_count'
 'dec_inflight'
@@ -8346,6 +8552,7 @@
 'decode_irq_flags'
 'decode_session4'
 'decode_session6'
+'decompress_method'
 'dec_pending'
 'decrypt_bh'
 'decrypt_blob'
@@ -8484,6 +8691,7 @@
 'delete_store'
 'del_gendisk'
 'del_mc_from_global_list'
+'del_named_trigger'
 'del_random_ready_callback'
 'del_timer'
 'del_timer_sync'
@@ -8528,6 +8736,8 @@
 'destroy_context_ldt'
 '__destroy_inode'
 'destroy_inode'
+'destroy_local_trace_kprobe'
+'destroy_local_trace_uprobe'
 'destroy_params'
 'destroy_sched_domain'
 'destroy_sched_domains_rcu'
@@ -8547,7 +8757,10 @@
 'detach_task_cfs_rq'
 'detect_extended_topology'
 'detect_extended_topology_early'
+'detect_ht'
+'detect_ht_early'
 'detect_intel_iommu'
+'detect_num_cpu_cores'
 'dev_activate'
 'dev_add_offload'
 'dev_add_pack'
@@ -8981,6 +9194,7 @@
 'devm_free_percpu'
 'devm_fwnode_gpiod_get_index'
 'devm_fwnode_pwm_get'
+'devm_gen_pool_create'
 'devm_get_clk_from_child'
 'devm_get_free_pages'
 'devm_gpiochip_add_data_with_key'
@@ -9029,6 +9243,15 @@
 'devm_input_device_match'
 'devm_input_device_release'
 'devm_input_device_unregister'
+'devm_ioport_map'
+'devm_ioport_unmap'
+'devm_ioremap'
+'devm_ioremap_release'
+'devm_ioremap_resource'
+'devm_ioremap_resource_wc'
+'devm_ioremap_uc'
+'devm_ioremap_wc'
+'devm_iounmap'
 '__devm_irq_alloc_descs'
 'devm_irq_desc_release'
 'devm_irq_match'
@@ -9076,6 +9299,7 @@
 'devm_nvmem_register'
 'devm_nvmem_release'
 'devm_nvmem_unregister'
+'devm_of_iomap'
 'devm_of_led_get'
 'devm_of_phy_get'
 'devm_of_phy_get_by_index'
@@ -9438,6 +9662,7 @@
 'die_cpus_show'
 '__die_header'
 'die_id_show'
+'digsig_verify'
 'dimmdev_ce_count_show'
 'dimmdev_dev_type_show'
 'dimmdev_edac_mode_show'
@@ -9501,6 +9726,7 @@
 'disable_smp'
 'disable_swap_slots_cache_lock'
 'disable_timer_pin_setup'
+'disable_trace_on_warning'
 'disable_TSC'
 'disable_write_same'
 'disable_write_zeroes'
@@ -10133,6 +10359,7 @@
 'dm_table_get_md_mempools'
 'dm_table_get_mode'
 'dm_table_get_num_targets'
+'dm_table_get_size'
 'dm_table_get_target'
 'dm_table_get_type'
 'dm_table_get_wildcard_target'
@@ -10209,6 +10436,7 @@
 'do_dw_dma_enable'
 'do_dw_dma_off'
 'do_dw_dma_on'
+'do_early_exception'
 'do_early_param'
 'do_emergency_remount'
 'do_emergency_remount_callback'
@@ -10264,6 +10492,7 @@
 'do_linkat'
 'do_lock_file_wait'
 '__do_loopback'
+'do_machine_check'
 'do_madvise'
 'domain_attach_iommu'
 'domain_context_clear_one_cb'
@@ -10307,6 +10536,8 @@
 '__do_notify'
 'do_notify_parent'
 'do_notify_parent_cldstop'
+'__do_once_done'
+'__do_once_start'
 'do_one_initcall'
 'do_one_pass'
 'do_open_execat'
@@ -10395,6 +10626,7 @@
 'do_symlinkat'
 'do_sync_core'
 'do_sync_work'
+'do_syscall_64'
 'do_sysctl_args'
 'do_sys_ftruncate'
 'do_syslog'
@@ -10500,6 +10732,9 @@
 'dqcache_shrink_count'
 'dqcache_shrink_scan'
 'dqget'
+'dql_completed'
+'dql_init'
+'dql_reset'
 'dqput'
 'dquot_acquire'
 'dquot_add_inodes'
@@ -10740,12 +10975,16 @@
 'dump_rules'
 'dump_schedule'
 'dump_skip'
+'dump_stack'
+'dump_stack_print_info'
+'dump_stack_set_arch_desc'
 'dump_task'
 'dump_truncate'
 'dump_unreclaimable_slab'
 'dump_user_range'
 'dump_vma_snapshot'
 'dup_fd'
+'dup_iter'
 'duplex_show'
 'dup_mm'
 'dup_userfaultfd'
@@ -10836,9 +11075,26 @@
 'dw_reg_write_swab'
 'dw_reg_write_word'
 'dx_probe'
+'dynamic_debug_exec_queries'
+'__dynamic_dev_dbg'
 'dynamic_dname'
 'dynamic_frame_show'
+'__dynamic_netdev_dbg'
+'__dynamic_pr_debug'
 'dyn_constraint'
+'dynevent_arg_add'
+'dynevent_arg_init'
+'dynevent_arg_pair_add'
+'dynevent_arg_pair_init'
+'dynevent_cmd_init'
+'dynevent_create'
+'dyn_event_register'
+'dyn_event_release'
+'dyn_event_seq_next'
+'dyn_event_seq_start'
+'dyn_event_seq_stop'
+'dyn_events_release_all'
+'dynevent_str_add'
 'dyntick_save_progress_counter'
 'e6xx_force_enable_hpet'
 'e820__end_of_low_ram_pfn'
@@ -10886,6 +11142,7 @@
 'early_acpi_osi_init'
 'early_alloc_pgt_buf'
 'early_au_setup'
+'early_cpu_init'
 'early_dbgp_init'
 'early_dbgp_write'
 'early_dump_pci_device'
@@ -10910,6 +11167,7 @@
 'early_iounmap'
 'early_irq_init'
 'early_is_amd_nb'
+'__early_make_pgtable'
 'early_memblock'
 'early_memremap'
 'early_memremap_pgprot_adjust'
@@ -10930,6 +11188,8 @@
 'early_serial8250_write'
 'early_serial_setup'
 '__early_set_fixmap'
+'early_setup_idt'
+'early_trace_init'
 'early_xdbc_parse_parameter'
 'early_xdbc_register_console'
 'early_xdbc_setup_hardware'
@@ -11465,6 +11725,7 @@
 'enqueue_timer'
 'enqueue_to_backlog'
 'enqueue_top_rt_rq'
+'enter_from_user_mode'
 'enter_lazy_tlb'
 'entropy_timer'
 'entry_attr_timeout'
@@ -11490,10 +11751,16 @@
 'erase_effect'
 'erase_locking_range'
 'err_broadcast'
+'errname'
 'errno_to_blk_status'
 'error'
 'errors_show'
 'errors_store'
+'err_pos'
+'errseq_check'
+'errseq_check_and_advance'
+'errseq_sample'
+'errseq_set'
 'err_threshold_show'
 'erst_clear'
 'erst_clearer'
@@ -11698,6 +11965,11 @@
 'evdev_write'
 'event_callback'
 'event_count_show'
+'event_enable_register_trigger'
+'event_enable_trigger_free'
+'event_enable_trigger_func'
+'event_enable_trigger_print'
+'event_enable_unregister_trigger'
 'eventfd_ctx_do_read'
 'eventfd_ctx_fdget'
 'eventfd_ctx_fileget'
@@ -11716,11 +11988,37 @@
 'events_ht_sysfs_show'
 'events_sysfs_show'
 'event_to_amd_uncore'
+'event_trace_add_tracer'
+'event_trace_del_tracer'
+'event_trace_init'
+'event_trigger_init'
+'event_triggers_call'
+'event_triggers_post_call'
 'evict'
 'evict_inodes'
 'exact_lock'
 'exact_match'
+'exc_alignment_check'
+'exc_bounds'
+'exc_coprocessor_error'
+'exc_coproc_segment_overrun'
+'exc_debug'
+'exc_device_not_available'
+'exc_divide_error'
+'exc_double_fault'
+'exc_general_protection'
 'exchange_tids'
+'exc_int3'
+'exc_invalid_op'
+'exc_invalid_tss'
+'exc_machine_check'
+'exc_nmi'
+'exc_overflow'
+'exc_page_fault'
+'exc_segment_not_present'
+'exc_simd_coprocessor_error'
+'exc_spurious_interrupt_bug'
+'exc_stack_segment'
 'execdomains_proc_show'
 'exec_mm_release'
 'execute_in_process_context'
@@ -11740,6 +12038,7 @@
 'ex_handler_wrmsr_unsafe'
 'exists_ordered_lsm'
 'exit_aio'
+'exit_amd_microcode'
 'exit_creds'
 'exit_files'
 'exit_fs'
@@ -11763,6 +12062,7 @@
 'exit_tasks_rcu_finish'
 'exit_tasks_rcu_start'
 'exit_thread'
+'exit_to_user_mode'
 'exit_to_user_mode_prepare'
 'expand_downwards'
 'expand_files'
@@ -12869,8 +13169,11 @@
 'fill_window'
 'filp_close'
 'filp_open'
+'filter_assign_type'
 'filter_chain'
+'filter_match_preds'
 'filter_mce'
+'filter_parse_regex'
 'final_ack_set'
 'finalize_and_send'
 'finalize_exec'
@@ -12881,10 +13184,12 @@
 'find_asymmetric_key'
 'find_autofs_mount'
 'find_battery'
+'find_bug'
 'find_busiest_group'
 'find_bus_resource_of_type'
 'find_candidate'
 'find_child_checks'
+'find_cpio_data'
 'find_css_set'
 'find_dependents_of'
 'find_devfreq_governor'
@@ -12895,10 +13200,14 @@
 'find_dump_kind'
 'find_entry_to_transmit'
 'find_equal_scalars'
+'__find_event_file'
+'find_event_file'
 'find_exception'
 'find_exported_symbol_in_section'
 'find_extend_vma'
+'find_first_bit'
 'find_first_swap'
+'find_first_zero_bit'
 'find_font'
 '_find_freq_ceil'
 'find_ge_pid'
@@ -12924,12 +13233,14 @@
 'find_inode_rcu'
 '__find_interface'
 'findintfep'
+'find_io_range_by_fwnode'
 'find_iova'
 'find_isa_irq_apic'
 'find_isa_irq_pin'
 'find_kallsyms_symbol'
 'find_keyring_by_name'
 'find_key_to_update'
+'find_last_bit'
 'find_later_rq'
 'find_lock_entry'
 'find_lock_later_rq'
@@ -12948,8 +13259,13 @@
 'find_min_pfn_with_active_regions'
 'find_module'
 'find_module_all'
+'find_named_trigger'
+'find_next_and_bit'
+'find_next_bit'
+'find_next_clump8'
 'find_next_id'
 'find_next_iomem_res'
+'find_next_zero_bit'
 'find_nls'
 '_find_opp_table'
 '_find_opp_table_unlocked'
@@ -13006,6 +13322,8 @@
 'firmware_init'
 'firmware_map_add_early'
 'firmware_map_add_entry'
+'firmware_map_add_hotplug'
+'firmware_map_remove'
 'firmware_memmap_init'
 'firmware_request_cache'
 'firmware_request_nowarn'
@@ -13029,6 +13347,7 @@
 '__fix_erratum_688'
 'fix_fullness_group'
 'fix_hypertransport_config'
+'fixup_bad_iret'
 'fixup_bpf_calls'
 'fixup_exception'
 'fixup_ht_bug'
@@ -13314,7 +13633,20 @@
 'fork_idle'
 'fork_init'
 'fork_usermode_driver'
+'fortify_panic'
 'fpregs_mark_activate'
+'fprop_fraction_percpu'
+'fprop_fraction_single'
+'fprop_global_destroy'
+'fprop_global_init'
+'__fprop_inc_percpu'
+'__fprop_inc_percpu_max'
+'__fprop_inc_single'
+'fprop_local_destroy_percpu'
+'fprop_local_destroy_single'
+'fprop_local_init_percpu'
+'fprop_local_init_single'
+'fprop_new_period'
 'fpstate_init'
 'fpstate_sanitize_xstate'
 'fpu__alloc_mathframe'
@@ -13405,6 +13737,7 @@
 'free_basic_memory_bitmaps'
 'free_bprm'
 'free_bridge'
+'free_bucket_spinlocks'
 'free_buffer'
 'free_buffer_head'
 'free_buffers'
@@ -13424,11 +13757,13 @@
 'free_dmar_iommu'
 'free_elfcorebuf'
 'free_equiv_cpu_table'
+'free_event_filter'
 'free_fair_sched_group'
 'free_fdtable_rcu'
 'free_fib_info'
 'free_fib_info_rcu'
 'free_fs_struct'
+'free_ftrace_func_mapper'
 'free_fw_priv'
 'free_hpage_workfn'
 'free_htab_elem'
@@ -13451,6 +13786,7 @@
 'free_ipc'
 'free_ipcs'
 'free_irq'
+'free_irq_cpu_rmap'
 'free_kernel_image_pages'
 'free_kthread_struct'
 'free_ldt_pgtables'
@@ -13784,6 +14120,97 @@
 'fs_umode_to_ftype'
 'fs_validate_description'
 'fsync_bdev'
+'ftrace_allocate_ftrace_ops'
+'ftrace_arch_code_modify_post_process'
+'ftrace_arch_code_modify_prepare'
+'ftrace_bug'
+'ftrace_clear_pids'
+'ftrace_create_filter_files'
+'ftrace_create_function_files'
+'ftrace_destroy_filter_files'
+'ftrace_destroy_function_files'
+'ftrace_disable_ftrace_graph_caller'
+'ftrace_dump'
+'ftrace_dyn_arch_init'
+'ftrace_enable_ftrace_graph_caller'
+'ftrace_enable_sysctl'
+'ftrace_event_is_function'
+'ftrace_filter_write'
+'ftrace_find_direct_func'
+'ftrace_find_event'
+'ftrace_find_rec_direct'
+'ftrace_free_filter'
+'ftrace_free_ftrace_ops'
+'ftrace_free_init_mem'
+'ftrace_free_mem'
+'ftrace_func_mapper_add_ip'
+'ftrace_func_mapper_find_ip'
+'ftrace_func_mapper_remove_ip'
+'ftrace_get_addr_curr'
+'ftrace_get_addr_new'
+'ftrace_graph_entry_stub'
+'ftrace_graph_exit_task'
+'ftrace_graph_get_ret_stack'
+'ftrace_graph_init_idle_task'
+'ftrace_graph_init_task'
+'ftrace_graph_is_dead'
+'ftrace_graph_ret_addr'
+'ftrace_graph_sleep_time_control'
+'ftrace_graph_stop'
+'ftrace_init'
+'ftrace_init_array_ops'
+'ftrace_init_global_array_ops'
+'ftrace_init_trace_array'
+'ftrace_init_tracefs'
+'ftrace_init_tracefs_toplevel'
+'ftrace_is_dead'
+'ftrace_kill'
+'ftrace_location'
+'ftrace_location_range'
+'ftrace_lookup_ip'
+'ftrace_make_call'
+'ftrace_make_nop'
+'ftrace_mod_address_lookup'
+'ftrace_mod_get_kallsym'
+'ftrace_modify_all_code'
+'ftrace_modify_call'
+'ftrace_modify_direct_caller'
+'ftrace_module_enable'
+'ftrace_module_init'
+'ftrace_notrace_write'
+'ftrace_now'
+'ftrace_ops_get_func'
+'ftrace_ops_set_global_filter'
+'ftrace_ops_test'
+'ftrace_ops_trampoline'
+'ftrace_pid_follow_fork'
+'ftrace_profile_free_filter'
+'ftrace_profile_set_filter'
+'ftrace_rec_iter_next'
+'ftrace_rec_iter_record'
+'ftrace_rec_iter_start'
+'ftrace_regex_open'
+'ftrace_regex_release'
+'ftrace_release_mod'
+'ftrace_replace_code'
+'ftrace_reset_array_ops'
+'ftrace_return_to_handler'
+'ftrace_run_stop_machine'
+'ftrace_set_clr_event'
+'ftrace_set_early_filter'
+'ftrace_set_filter'
+'ftrace_set_filter_ip'
+'ftrace_set_global_filter'
+'ftrace_set_global_notrace'
+'ftrace_set_notrace'
+'ftrace_shutdown'
+'ftrace_startup'
+'ftrace_test_record'
+'ftrace_text_reserved'
+'ftrace_update_ftrace_func'
+'ftrace_update_record'
+'__ftrace_vbprintk'
+'__ftrace_vprintk'
 'FUA_show'
 'full_bio_end_io'
 'full_name_hash'
@@ -13799,6 +14226,7 @@
 'func_id_show'
 'func_ptr_is_kernel_text'
 'func_states_equal'
+'function_graph_enter'
 'function_show'
 '__f_unlock_pos'
 'fup_on_ptw_show'
@@ -13836,6 +14264,7 @@
 'fuse_create_open'
 'fuse_ctl_add_conn'
 'fuse_ctl_add_dentry'
+'fuse_ctl_cleanup'
 'fuse_ctl_fill_super'
 'fuse_ctl_get_tree'
 'fuse_ctl_init'
@@ -14162,10 +14591,13 @@
 'gen9_stolen_size'
 'gen_codes'
 'generate_pm_trace'
+'generate_random_guid'
+'generate_random_uuid'
 'generic_access_phys'
 'generic_block_bmap'
 '__generic_block_fiemap'
 'generic_block_fiemap'
+'generic_bug_clear_once'
 'generic_check_addressable'
 'generic_cont_expand_simple'
 'generic_copy_file_range'
@@ -14305,6 +14737,34 @@
 'genphy_suspend'
 'genphy_update_link'
 'genphy_write_mmd_unsupported'
+'gen_pool_add_owner'
+'gen_pool_alloc_algo_owner'
+'gen_pool_avail'
+'gen_pool_best_fit'
+'gen_pool_create'
+'gen_pool_destroy'
+'gen_pool_dma_alloc'
+'gen_pool_dma_alloc_algo'
+'gen_pool_dma_alloc_align'
+'gen_pool_dma_zalloc'
+'gen_pool_dma_zalloc_algo'
+'gen_pool_dma_zalloc_align'
+'gen_pool_first_fit'
+'gen_pool_first_fit_align'
+'gen_pool_first_fit_order_align'
+'gen_pool_fixed_alloc'
+'gen_pool_for_each_chunk'
+'gen_pool_free_owner'
+'gen_pool_get'
+'gen_pool_has_addr'
+'gen_pool_set_algo'
+'gen_pool_size'
+'gen_pool_virt_to_phys'
+'__genradix_free'
+'__genradix_iter_peek'
+'__genradix_prealloc'
+'__genradix_ptr'
+'__genradix_ptr_alloc'
 'genregs_get'
 'genregs_set'
 'gen_replace_estimator'
@@ -14345,8 +14805,11 @@
 'get_color'
 'getconsxy'
 'get_cont_catch'
+'get_cpu_address_sizes'
 'get_cpu_cacheinfo'
+'get_cpu_cap'
 'get_cpu_device'
+'get_cpu_entry_area'
 'get_cpu_idle_time'
 'get_cpu_idle_time_us'
 'get_cpu_iowait_time_us'
@@ -14399,6 +14862,7 @@
 'get_iface_id'
 'get_image_page'
 'get_info'
+'get_injectable_error_type'
 '__get_insn_slot'
 'get_instantiation_keyring'
 'get_io_context'
@@ -14435,6 +14899,7 @@
 'get_name'
 'getname'
 '__get_name_cell'
+'get_named_trigger_data'
 'getname_flags'
 'getname_kernel'
 'get_net_ns'
@@ -14448,6 +14913,7 @@
 'get_next_timer_interrupt'
 'get_nohz_timer_target'
 'get_nr_dirty_inodes'
+'get_obj_cgroup_from_current'
 'get_old_itimerspec32'
 'get_old_pud'
 'get_old_timespec32'
@@ -14455,6 +14921,8 @@
 '_get_opp_count'
 '_get_opp_table_kref'
 'get_optimized_kprobe'
+'get_option'
+'get_options'
 'get_origin_minimum_chunksize'
 'get_orlov_stats'
 'get_page_from_freelist'
@@ -14505,6 +14973,7 @@
 'get_slot_status'
 'get_srh'
 'get_stack_info'
+'get_stack_info_noinstr'
 'get_state_synchronize_rcu'
 'get_status'
 'get_super'
@@ -14514,6 +14983,7 @@
 'get_swap_page_of_type'
 'get_swap_pages'
 'get_symbol_pos'
+'get_syscall_name'
 'get_taint'
 'get_target_type'
 'get_target_version'
@@ -14609,6 +15079,7 @@
 'give_up_console'
 'global_cache_flush'
 'global_dirty_limits'
+'glob_match'
 'glp_get_event_constraints'
 'glue_cbc_decrypt_req_128bit'
 'glue_cbc_encrypt_req_128bit'
@@ -14773,6 +15244,8 @@
 'grab_cache_page_write_begin'
 'grab_super'
 'graft_tree'
+'graph_trace_close'
+'graph_trace_open'
 'gred_change'
 'gred_change_table_def'
 'gred_dequeue'
@@ -14814,6 +15287,10 @@
 'group_store'
 'grow_tree_refs'
 'guard_bio_eod'
+'guid_gen'
+'guid_parse'
+'__gunzip'
+'gunzip'
 '__gup_longterm_locked'
 'haltpoll_cpu_offline'
 'haltpoll_cpu_online'
@@ -14873,6 +15350,7 @@
 'hash_accept_nokey'
 'hash_accept_parent'
 'hash_accept_parent_nokey'
+'hash_and_copy_to_iter'
 'hash_bind'
 'hash_futex'
 'hashlen_string'
@@ -14976,7 +15454,10 @@
 'hest_parse_cmc'
 'hest_parse_ghes'
 'hest_parse_ghes_count'
+'hex2bin'
 'hexdump'
+'hex_dump_to_buffer'
+'hex_to_bin'
 'hfsc_bind_tcf'
 'hfsc_change_class'
 'hfsc_change_qdisc'
@@ -15226,6 +15707,7 @@
 'hpet_poll'
 '_hpet_print_config'
 'hpet_read'
+'hpet_readl'
 'hpet_register_irq_handler'
 'hpet_release'
 'hpet_reserve_platform_timers'
@@ -15272,6 +15754,11 @@
 'hrtimer_start_range_ns'
 'hrtimer_try_to_cancel'
 'hrtimer_wakeup'
+'hsiphash_1u32'
+'hsiphash_2u32'
+'hsiphash_3u32'
+'hsiphash_4u32'
+'__hsiphash_aligned'
 'hsu_dma_chan_start'
 'hsu_dma_desc_free'
 'hsu_dma_do_irq'
@@ -15822,6 +16309,11 @@
 'icmpv6_sk_init'
 'icmpv6_xrlim_allow'
 'icq_free_icq_rcu'
+'ida_alloc_range'
+'ida_destroy'
+'ida_free'
+'identify_boot_cpu'
+'identify_secondary_cpu'
 'identity_show'
 'ident_p4d_init'
 'ident_pmd_init'
@@ -15850,7 +16342,19 @@
 'idma32_set_device_name'
 'idma32_suspend_chan'
 'idProduct_show'
+'idr_alloc'
+'idr_alloc_cyclic'
+'idr_alloc_u32'
 'idr_callback'
+'idr_destroy'
+'idr_find'
+'idr_for_each'
+'idr_get_free'
+'idr_get_next'
+'idr_get_next_ul'
+'idr_preload'
+'idr_remove'
+'idr_replace'
 'id_show'
 'idt_invalidate'
 'idt_setup_apic_and_irq_gates'
@@ -16040,6 +16544,7 @@
 'ima_release_policy'
 'ima_restore_measurement_entry'
 'ima_restore_measurement_list'
+'ima_should_queue_key'
 'ima_show_htable_violations'
 'ima_show_measurements_count'
 'ima_show_template_buf'
@@ -16062,6 +16567,9 @@
 'im_explorer_detect'
 'imp_def_interrupts_show'
 'implement'
+'__import_iovec'
+'import_iovec'
+'import_single_range'
 'in4_pton'
 'in6_dev_finish_destroy'
 'in6_dev_finish_destroy_rcu'
@@ -16091,6 +16599,7 @@
 'in_dev_finish_destroy'
 'in_dev_rcu_put'
 'in_egroup_p'
+'in_entry_stack'
 'inet4_pton'
 'inet6_add_offload'
 'inet6_add_protocol'
@@ -16374,6 +16883,7 @@
 'init_cpu_online'
 'init_cpu_possible'
 'init_cpu_present'
+'init_currently_empty_zone'
 'init_dax_wait_table'
 'init_debug_store_on_cpu'
 'init_default_flow_dissectors'
@@ -16408,6 +16918,8 @@
 'init_freq_invariance'
 'init_freq_invariance_cppc'
 'init_frontswap'
+'init_ftrace_syscalls'
+'init_function_trace'
 'init_fw'
 'init_hugetlbfs_fs'
 'init_hw_breakpoint'
@@ -16464,6 +16976,7 @@
 'init_pcmcia_bus'
 'init_pcmcia_cs'
 'init_peercred'
+'init_per_zone_wmark_min'
 'init_pipe_fs'
 'init_pkru_read_file'
 'init_pkru_write_file'
@@ -16533,6 +17046,7 @@
 'init_zero_pfn'
 'init_zhaoxin'
 'init_zswap'
+'in_lock_functions'
 '__inode_add_bytes'
 'inode_add_bytes'
 'inode_add_lru'
@@ -16772,6 +17286,7 @@
 'instance_ue_count_show'
 'int340x_thermal_handler_attach'
 'int3_exception_notify'
+'in_task_stack'
 'integral_cutoff_show'
 'integral_cutoff_store'
 'integrity_add_key'
@@ -17129,6 +17644,10 @@
 'interval_iter_next'
 'interval_show'
 'interval_subtree_search'
+'interval_tree_insert'
+'interval_tree_iter_first'
+'interval_tree_iter_next'
+'interval_tree_remove'
 'intf_assoc_attrs_are_visible'
 'int_pln_enable_setup'
 'int_poll_timeout'
@@ -17486,6 +18005,8 @@
 'io_poll_rewait'
 'io_poll_task_func'
 'io_poll_wake'
+'ioport_map'
+'ioport_unmap'
 'io_prep_async_work'
 'io_prep_rw'
 'ioprio_best'
@@ -17498,6 +18019,19 @@
 '__io_queue_sqe'
 'io_queue_sqe'
 'io_read'
+'ioread16'
+'ioread16be'
+'ioread16_rep'
+'ioread32'
+'ioread32be'
+'__ioread32_copy'
+'ioread32_rep'
+'ioread64be_hi_lo'
+'ioread64be_lo_hi'
+'ioread64_hi_lo'
+'ioread64_lo_hi'
+'ioread8'
+'ioread8_rep'
 'io_recv'
 'io_recvmsg'
 '__io_recvmsg_copy_hdr'
@@ -17592,6 +18126,24 @@
 'iova_cache_put'
 'iova_domain_flush'
 'iova_insert_rbtree'
+'iovec_from_user'
+'iov_iter_advance'
+'iov_iter_alignment'
+'iov_iter_bvec'
+'iov_iter_copy_from_user_atomic'
+'iov_iter_discard'
+'iov_iter_fault_in_readable'
+'iov_iter_for_each_range'
+'iov_iter_gap_alignment'
+'iov_iter_get_pages'
+'iov_iter_get_pages_alloc'
+'iov_iter_init'
+'iov_iter_kvec'
+'iov_iter_npages'
+'iov_iter_pipe'
+'iov_iter_revert'
+'iov_iter_single_seg_count'
+'iov_iter_zero'
 'io_wake_function'
 'io_watchdog_func'
 'io_worker_handle_work'
@@ -17616,6 +18168,20 @@
 'io_wq_worker_sleeping'
 'io_wq_worker_wake'
 'io_write'
+'iowrite16'
+'iowrite16be'
+'iowrite16_rep'
+'iowrite32'
+'iowrite32be'
+'__iowrite32_copy'
+'iowrite32_rep'
+'iowrite64be_hi_lo'
+'iowrite64be_lo_hi'
+'__iowrite64_copy'
+'iowrite64_hi_lo'
+'iowrite64_lo_hi'
+'iowrite8'
+'iowrite8_rep'
 '__ip4_datagram_connect'
 'ip4_datagram_connect'
 'ip4_datagram_release_cb'
@@ -18289,6 +18855,7 @@
 'irq_complete_move'
 'irq_cpu_offline'
 'irq_cpu_online'
+'irq_cpu_rmap_add'
 'irq_create_affinity_masks'
 'irq_create_direct_mapping'
 'irq_create_fwspec_mapping'
@@ -18346,7 +18913,13 @@
 'irq_enable'
 'irq_enter'
 'irq_enter_rcu'
+'irqentry_enter'
+'irqentry_enter_from_user_mode'
+'irqentry_exit'
 'irqentry_exit_cond_resched'
+'irqentry_exit_to_user_mode'
+'irqentry_nmi_enter'
+'irqentry_nmi_exit'
 'irq_exit'
 'irq_exit_rcu'
 'irq_find_mapping'
@@ -18399,6 +18972,11 @@
 'irq_pm_install_action'
 'irq_pm_remove_action'
 'irq_pm_syscore_resume'
+'irq_poll_complete'
+'irq_poll_disable'
+'irq_poll_enable'
+'irq_poll_init'
+'irq_poll_sched'
 'irqpoll_setup'
 '__irq_put_desc_unlock'
 'irqrouter_resume'
@@ -18444,6 +19022,13 @@
 'irq_wait_for_poll'
 '__irq_wake_thread'
 'irq_wake_thread'
+'irq_work_needs_cpu'
+'irq_work_queue'
+'irq_work_queue_on'
+'irq_work_run'
+'irq_work_single'
+'irq_work_sync'
+'irq_work_tick'
 'ir_raw_edge_handle'
 'ir_raw_encode_carrier'
 'ir_raw_encode_scancode'
@@ -18491,6 +19076,7 @@
 'is_empty_dir_inode'
 'is_file_shm_hugepages'
 'is_free_buddy_page'
+'is_ftrace_trampoline'
 'is_hardlockup'
 'is_hash_blacklisted'
 'is_hibernate_resume_dev'
@@ -18513,6 +19099,7 @@
 'is_module_percpu_address'
 'is_module_sig_enforced'
 'is_module_text_address'
+'is_named_trigger'
 '__isolate_free_page'
 'isolate_freepages_block'
 'isolate_freepages_range'
@@ -18537,8 +19124,10 @@
 'is_swbp_insn'
 'is_swiotlb_active'
 'is_test_sglist_corrupted'
+'is_tracing_stopped'
 'is_transparent_hugepage'
 'is_trap_insn'
+'is_valid_bugaddr'
 'is_valid_gup_flags'
 'is_virtio_device'
 'is_visible'
@@ -18755,6 +19344,7 @@
 'kallsyms_show_value'
 'karma_partition'
 'k_ascii'
+'kasprintf'
 'kauditd_hold_skb'
 'kauditd_rehold_skb'
 'kauditd_retry_skb'
@@ -18816,6 +19406,8 @@
 'kernel_page_present'
 'kernel_param_lock'
 'kernel_param_unlock'
+'kernel_physical_mapping_change'
+'kernel_physical_mapping_init'
 'kernel_power_off'
 '__kernel_read'
 'kernel_read'
@@ -19051,6 +19643,28 @@
 'key_user_lookup'
 'key_user_put'
 'key_validate'
+'__kfifo_alloc'
+'__kfifo_dma_in_finish_r'
+'__kfifo_dma_in_prepare'
+'__kfifo_dma_in_prepare_r'
+'__kfifo_dma_out_finish_r'
+'__kfifo_dma_out_prepare'
+'__kfifo_dma_out_prepare_r'
+'__kfifo_free'
+'__kfifo_from_user'
+'__kfifo_from_user_r'
+'__kfifo_in'
+'__kfifo_init'
+'__kfifo_in_r'
+'__kfifo_len_r'
+'__kfifo_max_r'
+'__kfifo_out'
+'__kfifo_out_peek'
+'__kfifo_out_peek_r'
+'__kfifo_out_r'
+'__kfifo_skip_r'
+'__kfifo_to_user'
+'__kfifo_to_user_r'
 'k_fn'
 'kfree'
 'kfree_const'
@@ -19068,6 +19682,7 @@
 'kfree_skb_list'
 'kfree_skbmem'
 'kfree_skb_partial'
+'kfree_strarray'
 'kgdb_arch_exit'
 'kgdb_arch_handle_exception'
 'kgdb_arch_init'
@@ -19192,12 +19807,25 @@
 'k_i_store'
 'k_itimer_rcu_free'
 'kjournald2'
+'klist_add_before'
+'klist_add_behind'
+'klist_add_head'
+'klist_add_tail'
 'klist_children_get'
 'klist_children_put'
 'klist_class_dev_get'
 'klist_class_dev_put'
+'klist_del'
 'klist_devices_get'
 'klist_devices_put'
+'klist_init'
+'klist_iter_exit'
+'klist_iter_init'
+'klist_iter_init_node'
+'klist_next'
+'klist_node_attached'
+'klist_prev'
+'klist_remove'
 'k_lock'
 'k_lowercase'
 '__kmalloc'
@@ -19228,6 +19856,7 @@
 'kmem_cache_shrink'
 '__kmem_cache_shutdown'
 'kmem_cache_size'
+'kmemdup'
 'kmemdup_nul'
 'k_meta'
 'km_get_page'
@@ -19263,9 +19892,37 @@
 'knl_get_aperf_mperf_shift'
 'knl_get_turbo_pstate'
 'known_siginfo_layout'
+'kobj_child_ns_ops'
+'kobject_add'
+'kobject_create'
+'kobject_create_and_add'
+'kobject_del'
+'kobject_get'
+'kobject_get_ownership'
+'kobject_get_path'
+'kobject_get_unless_zero'
+'kobject_init'
+'kobject_init_and_add'
+'kobject_move'
+'kobject_namespace'
+'kobject_put'
+'kobject_rename'
+'kobject_set_name'
+'kobject_set_name_vargs'
+'kobject_synth_uevent'
+'kobject_uevent'
+'kobject_uevent_env'
 'kobj_lookup'
 'kobj_map'
 'kobj_map_init'
+'kobj_ns_current_may_mount'
+'kobj_ns_drop'
+'kobj_ns_grab_current'
+'kobj_ns_initial'
+'kobj_ns_netlink'
+'kobj_ns_ops'
+'kobj_ns_type_register'
+'kobj_ns_type_registered'
 'kobj_unmap'
 'k_pad'
 'kpagecgroup_read'
@@ -19284,6 +19941,10 @@
 'kprobe_busy_end'
 'kprobe_cache_get_kallsym'
 'kprobe_debug_handler'
+'__kprobe_event_add_fields'
+'kprobe_event_cmd_init'
+'kprobe_event_delete'
+'__kprobe_event_gen_cmd_start'
 'kprobe_exceptions_notify'
 'kprobe_fault_handler'
 'kprobe_flush_task'
@@ -19306,6 +19967,11 @@
 'krealloc'
 '__kretprobe_trampoline_handler'
 'k_self'
+'kset_create_and_add'
+'kset_find_obj'
+'kset_init'
+'kset_register'
+'kset_unregister'
 'k_shift'
 '__ksize'
 'ksize'
@@ -19325,7 +19991,32 @@
 'KSTK_ESP'
 'kstrdup'
 'kstrdup_const'
+'kstrdup_quotable'
+'kstrdup_quotable_cmdline'
+'kstrdup_quotable_file'
 'kstrndup'
+'kstrtobool'
+'kstrtobool_from_user'
+'kstrtoint'
+'kstrtoint_from_user'
+'_kstrtol'
+'kstrtol_from_user'
+'kstrtoll'
+'kstrtoll_from_user'
+'kstrtos16'
+'kstrtos16_from_user'
+'kstrtos8'
+'kstrtos8_from_user'
+'kstrtou16'
+'kstrtou16_from_user'
+'kstrtou8'
+'kstrtou8_from_user'
+'kstrtouint'
+'kstrtouint_from_user'
+'_kstrtoul'
+'kstrtoul_from_user'
+'kstrtoull'
+'kstrtoull_from_user'
 'kswapd'
 'kswapd_init'
 'kswapd_run'
@@ -19400,6 +20091,7 @@
 'kthread_worker_fn'
 'ktime_add_safe'
 'ktime_get'
+'ktime_get_boot_fast_ns'
 'ktime_get_coarse_real_ts64'
 'ktime_get_coarse_ts64'
 'ktime_get_coarse_with_offset'
@@ -19409,6 +20101,7 @@
 'ktime_get_raw_fast_ns'
 'ktime_get_raw_ts64'
 'ktime_get_real_fast_ns'
+'__ktime_get_real_seconds'
 'ktime_get_real_seconds'
 'ktime_get_real_ts64'
 'ktime_get_resolution_ns'
@@ -19420,6 +20113,8 @@
 'ktime_mono_to_any'
 'kt_serial_in'
 'kt_serial_setup'
+'kvasprintf'
+'kvasprintf_const'
 'kvfree'
 'kvfree_call_rcu'
 'kvfree_rcu_local'
@@ -19431,6 +20126,8 @@
 'kvm_arch_para_hints'
 'kvm_async_pf_task_wait_schedule'
 'kvm_async_pf_task_wake'
+'kvm_check_and_clear_guest_paused'
+'kvmclock_init'
 'kvm_cpu_down_prepare'
 '__kvm_cpuid_base'
 'kvm_cpu_online'
@@ -19441,12 +20138,14 @@
 'kvm_flush_tlb_others'
 'kvm_guest_cpu_init'
 'kvm_guest_init'
+'__kvm_handle_async_pf'
 'kvm_init_platform'
 'kvm_io_delay'
 'kvm_msi_ext_dest_id'
 'kvm_para_available'
 'kvm_pv_guest_cpu_reboot'
 'kvm_pv_reboot_notify'
+'kvm_read_and_reset_apf_flags'
 'kvm_send_ipi_mask'
 'kvm_send_ipi_mask_allbutself'
 'kvm_set_posted_intr_wakeup_handler'
@@ -19556,6 +20255,7 @@
 'late_resume_init'
 'layout_show'
 'layout_store'
+'lbr_from_signext_quirk_wr'
 'lbr_is_visible'
 'lcm'
 'lcm_not_zero'
@@ -19629,6 +20329,7 @@
 'led_trigger_unregister_simple'
 'led_trigger_write'
 'ledtrig_panic_init'
+'ledtrig_usb_exit'
 'ledtrig_usb_init'
 'led_update_brightness'
 'led_work'
@@ -19657,6 +20358,7 @@
 'libata_trace_parse_qc_flags'
 'libata_trace_parse_status'
 'libata_trace_parse_subcmd'
+'libata_transport_exit'
 'libata_transport_init'
 'lifebook_absolute_mode'
 'lifebook_detect'
@@ -19673,6 +20375,14 @@
 'linear_iterate_devices'
 'linear_map'
 'linear_prepare_ioctl'
+'linear_range_get_max_value'
+'linear_range_get_selector_high'
+'linear_range_get_selector_low'
+'linear_range_get_selector_low_array'
+'linear_range_get_value'
+'linear_range_get_value_array'
+'linear_range_values_in_range'
+'linear_range_values_in_range_array'
 'linear_report_zones'
 'linear_status'
 'lineevent_free'
@@ -19731,6 +20441,7 @@
 'lirc_bpf_free'
 'lirc_bpf_run'
 'lirc_close'
+'lirc_dev_exit'
 'lirc_dev_init'
 'lirc_ioctl'
 'lirc_mode2_func_proto'
@@ -19748,6 +20459,8 @@
 'lirc_transmit'
 'lirc_unregister'
 'list_add_sorted'
+'__list_add_valid'
+'__list_del_entry_valid'
 'list_devices'
 'list_get_page'
 'list_locations'
@@ -19765,20 +20478,26 @@
 'list_lru_walk_one_irq'
 'list_netdevice'
 'list_next_page'
+'list_sort'
 'list_version_get_info'
 'list_version_get_needed'
 '__list_versions'
 'list_versions'
 'listxattr'
 'll_back_merge_fn'
+'llist_add_batch'
+'llist_del_first'
+'llist_reverse_order'
 'll_rw_block'
 'lmce_supported'
 'loadavg_proc_show'
 'load_balance'
 'load_builtin_intel_microcode'
 'load_current_idt'
+'load_direct_gdt'
 'load_elf_binary'
 'load_elf_phdrs'
+'load_fixmap_gdt'
 'load_image'
 'load_image_and_restore'
 'load_image_lzo'
@@ -19789,6 +20508,7 @@
 'load_msg'
 'load_nls'
 'load_nls_default'
+'load_percpu_segment'
 'load_ramdisk'
 'load_script'
 'load_system_certificate_list'
@@ -19829,6 +20549,14 @@
 'lock_page_memcg'
 '__lock_page_or_retry'
 'lock_rdev'
+'lockref_get'
+'lockref_get_not_dead'
+'lockref_get_not_zero'
+'lockref_get_or_lock'
+'lockref_mark_dead'
+'lockref_put_not_zero'
+'lockref_put_or_lock'
+'lockref_put_return'
 'lock_rename'
 'locks_alloc_lock'
 'locks_check_ctx_file_list'
@@ -19884,6 +20612,11 @@
 'log_buf_vmcoreinfo_setup'
 '__log_error'
 'logfc'
+'logic_pio_register_range'
+'logic_pio_to_hwaddr'
+'logic_pio_trans_cpuaddr'
+'logic_pio_trans_hwaddr'
+'logic_pio_unregister_range'
 'loglevel'
 'log_non_standard_event'
 'logon_vet_description'
@@ -19907,6 +20640,7 @@
 'lookup_module_symbol_name'
 'lookup_mountpoint'
 'lookup_nulls_elem_raw'
+'look_up_OID'
 'lookup_one_len'
 'lookup_one_len_common'
 'lookup_one_len_unlocked'
@@ -20056,6 +20790,7 @@
 'mach_reboot_fixups'
 'mach_set_rtc_mmss'
 'mac_partition'
+'mac_pton'
 'madvise_cold'
 'madvise_cold_or_pageout_pte_range'
 'madvise_free_huge_pmd'
@@ -20150,6 +20885,7 @@
 'mark_reg_unknown'
 'mark_rodata_ro'
 'mark_tsc_async_resets'
+'mark_tsc_unstable'
 'mark_wake_futex'
 'mark_wakeup_next_waiter'
 'mask_8259A'
@@ -20169,11 +20905,20 @@
 'match_exception_partial'
 'match_fanout_group'
 'match_file'
+'match_hex'
 'match_hid_uid'
+'match_int'
 'match_location'
+'match_octal'
 'match_pci_dev'
 'match_pci_dev_by_id'
 'match_revfn'
+'match_strdup'
+'match_string'
+'match_strlcpy'
+'match_token'
+'match_u64'
+'match_wildcard'
 'math_error'
 'max_active_show'
 'max_active_store'
@@ -20323,6 +21068,7 @@
 'mce_read_aux'
 'mce_register_decode_chain'
 'mce_register_injector_chain'
+'mce_setup'
 'mce_severity_amd'
 'mce_severity_intel'
 'mce_syscore_resume'
@@ -20555,6 +21301,8 @@
 'membarrier_private_expedited'
 'membarrier_register_private_expedited'
 'membarrier_update_current_mm'
+'memblock_add'
+'memblock_add_node'
 'memblock_alloc_exact_nid_raw'
 'memblock_alloc_internal'
 'memblock_alloc_range_nid'
@@ -20562,15 +21310,41 @@
 'memblock_alloc_try_nid_raw'
 'memblock_allow_resize'
 'memblock_cap_memory_range'
+'memblock_clear_hotplug'
+'memblock_clear_nomap'
 'memblock_discard'
+'memblock_dump_all'
+'memblock_end_of_DRAM'
 'memblock_enforce_memory_limit'
 'memblock_find_dma_reserve'
+'memblock_find_in_range'
+'memblock_free'
 'memblock_free_all'
 '__memblock_free_late'
 'memblock_free_pages'
+'memblock_get_current_limit'
+'memblock_is_map_memory'
+'memblock_is_memory'
+'memblock_is_region_memory'
+'memblock_is_region_reserved'
+'memblock_is_reserved'
+'memblock_mark_hotplug'
+'memblock_mark_mirror'
+'memblock_mark_nomap'
 'memblock_mem_limit_remove_map'
+'memblock_overlaps_region'
 'memblock_phys_alloc_range'
 'memblock_phys_alloc_try_nid'
+'memblock_phys_mem_size'
+'memblock_remove'
+'memblock_reserve'
+'memblock_reserved_size'
+'memblock_search_pfn_nid'
+'memblock_set_current_limit'
+'memblock_set_node'
+'memblock_start_of_DRAM'
+'memblock_trim_memory'
+'__memcat_p'
 'memcg_alloc_page_obj_cgroups'
 'memcg_drain_all_list_lrus'
 'memcg_event_ptable_queue_proc'
@@ -20679,7 +21453,11 @@
 'memcg_to_vmpressure'
 'memcg_update_all_list_lrus'
 'memcg_write_event_control'
+'memchr'
+'memchr_inv'
+'memcmp'
 'memcmp_pages'
+'memcpy'
 'memcpy_count_show'
 '__memcpy_flushcache'
 'memcpy_fromio'
@@ -20696,7 +21474,10 @@
 'mem_init_print_info'
 'mem_lseek'
 'memmap_attr_show'
+'memmap_init'
+'memmap_init_zone'
 'mem_map_via_hcall'
+'memmove'
 'mem_open'
 'memory_block_size_bytes'
 'memory_bm_clear_bit'
@@ -20728,6 +21509,7 @@
 'memory_stat_format'
 'memory_stat_show'
 'memory_stats_init'
+'memparse'
 'mempool_alloc'
 'mempool_alloc_pages'
 'mempool_alloc_pages_isa'
@@ -20748,9 +21530,11 @@
 'mem_region_callback'
 'mem_release'
 'memremap'
+'memscan'
 'mem_section_usage_size'
 'mem_serial_in'
 'mem_serial_out'
+'memset'
 'memset_io'
 'mem_sleep_default_setup'
 'mem_sleep_show'
@@ -20774,6 +21558,7 @@
 'memtype_seq_start'
 'memtype_seq_stop'
 'memunmap'
+'memweight'
 'mem_write'
 'menu_enable_device'
 'menu_reflect'
@@ -20856,6 +21641,7 @@
 'mfill_zeropage'
 'mfld_get_clk_rate_khz'
 'mfld_setup'
+'microcode_check'
 'microcode_fini_cpu_amd'
 'microcode_init'
 'microcode_init_cpu'
@@ -20899,6 +21685,8 @@
 'mini_qdisc_pair_init'
 'mini_qdisc_pair_swap'
 'mini_qdisc_rcu_func'
+'minmax_running_max'
+'minmax_running_min'
 'min_partial_show'
 'min_partial_store'
 'min_ratio_show'
@@ -21019,6 +21807,7 @@
 'mm_find_pmd'
 'mm_get_huge_zero_page'
 'mm_init'
+'mminit_validate_memmodel_limits'
 'mminit_verify_pageflags_layout'
 'mminit_verify_zonelist'
 '__mm_populate'
@@ -21092,6 +21881,7 @@
 'mode_show'
 'mode_store'
 'mode_string'
+'modify_ftrace_direct'
 'modify_prefix_route'
 'modify_user_hw_breakpoint'
 'modify_user_hw_breakpoint_check'
@@ -21116,6 +21906,8 @@
 'module_arch_freeing_init'
 'module_attr_show'
 'module_attr_store'
+'module_bug_cleanup'
+'module_bug_finalize'
 'module_event'
 'module_exit_section'
 'module_finalize'
@@ -21627,6 +22419,8 @@
 'native_apic_icr_read'
 'native_apic_icr_write'
 'native_apic_wait_icr_idle'
+'native_calibrate_cpu_early'
+'native_calibrate_tsc'
 'native_cpu_die'
 'native_cpu_disable'
 'native_cpu_up'
@@ -21648,7 +22442,11 @@
 'native_play_dead'
 'native_pv_lock_init'
 'native_restore_boot_irq_mode'
+'native_restore_fl'
 'native_safe_apic_wait_icr_idle'
+'native_save_fl'
+'native_sched_clock'
+'native_sched_clock_from_tsc'
 'native_send_call_func_ipi'
 'native_send_call_func_single_ipi'
 '__native_set_fixmap'
@@ -21660,6 +22458,8 @@
 'native_steal_clock'
 'native_stop_other_cpus'
 'native_tss_update_io_bitmap'
+'native_write_cr0'
+'native_write_cr4'
 'ncpus_cmp_func'
 'ncsi_add_channel'
 'ncsi_add_package'
@@ -22268,6 +23068,7 @@
 'newseg'
 'new_sync_read'
 'new_sync_write'
+'next_arg'
 'next_bits'
 'next_group'
 'nexthop_alloc'
@@ -22282,7 +23083,10 @@
 '__nexthop_replace_notify'
 'nexthop_select_path'
 'nexthop_set_hw_flags'
+'__next_mem_pfn_range'
 '__next_mem_range'
+'__next_mem_range_rev'
+'__next_node_in'
 'next_online_pgdat'
 'next_queue_may_preempt'
 'next_signal'
@@ -22358,18 +23162,44 @@
 'nhm_limit_period'
 'nh_netdev_event'
 'nh_valid_get_del_req'
+'nla_append'
+'nla_find'
+'nla_get_range_signed'
+'nla_get_range_unsigned'
+'nla_memcmp'
+'nla_memcpy'
+'__nla_parse'
+'nla_policy_len'
+'__nla_put'
+'nla_put'
+'__nla_put_64bit'
+'nla_put_64bit'
 'nla_put_ifalias'
+'__nla_put_nohdr'
+'nla_put_nohdr'
+'__nla_reserve'
+'nla_reserve'
+'__nla_reserve_64bit'
+'nla_reserve_64bit'
+'__nla_reserve_nohdr'
+'nla_reserve_nohdr'
+'nla_strcmp'
+'nla_strdup'
+'nla_strscpy'
+'__nla_validate'
 'nl_fib_input'
 'nl_fib_lookup'
 'nlmsg_notify'
 'nlmsg_populate_fdb'
 '__nlmsg_put'
+'nmi_cpu_backtrace'
 'nmi_cpu_backtrace_handler'
 'nmi_handle'
 'nmi_panic'
 'nmi_panic_self_stop'
 'nmi_raise_cpu_backtrace'
 'nmi_shootdown_cpus'
+'nmi_trigger_cpumask_backtrace'
 'nmi_uaccess_okay'
 'nmi_warning_debugfs'
 'n_null_close'
@@ -22398,6 +23228,8 @@
 '_nohz_idle_balance'
 'no_initrd'
 'noirqdebug_setup'
+'noist_exc_debug'
+'noist_exc_machine_check'
 'no_llseek'
 'non_ehci_add'
 'nonmi_ipi_setup'
@@ -22455,9 +23287,11 @@
 'notifier_chain_register'
 'notify_change'
 'notify_cpu_starting'
+'notify_die'
 'notify_rule_change'
 'notify_user_space'
 'notimercheck'
+'notsc_setup'
 'notsupp_get_next_key'
 'no_tty'
 'not_visible'
@@ -22483,12 +23317,14 @@
 'nr_overcommit_hugepages_store'
 'nr_processes'
 'nr_running'
+'ns2usecs'
 'ns_capable'
 'ns_capable_noaudit'
 'ns_capable_setid'
 'ns_dname'
 'nsecs_to_jiffies'
 'nsecs_to_jiffies64'
+'nsecs_to_usecs'
 'nsec_to_clock_t'
 'nsfs_evict'
 'nsfs_init'
@@ -22556,6 +23392,7 @@
 'number_of_sets_show'
 'num_digits'
 'num_pages_show'
+'num_to_str'
 'num_users_show'
 'nvbridge_check_legacy_irq_routing'
 'nvenet_msi_disable'
@@ -23043,6 +23880,8 @@
 'parse_freebsd'
 'parse_hardened_usercopy'
 'parse_header'
+'_parse_integer'
+'_parse_integer_fixup_radix'
 'parse_lapic'
 'parse_lapic_timer_c2_ok'
 'parse_memmap_opt'
@@ -23069,6 +23908,7 @@
 'parse_no_stealacc'
 'parse_openbsd'
 'parse_options'
+'parse_option_str'
 'parse_pcc_subspace'
 'parse_pci'
 'parse_pmtmr'
@@ -23134,6 +23974,7 @@
 'pat_pagerange_is_ram'
 'pat_pfn_immune_to_uc_mtrr'
 'pause_fill_reply'
+'pause_named_trigger'
 'pause_prepare_data'
 'pause_reply_size'
 'pbus_size_mem'
@@ -23647,10 +24488,15 @@
 'pci_intx'
 'pci_invalid_bar'
 'pci_ioapic_remove'
+'pci_iomap'
+'pci_iomap_range'
+'pci_iomap_wc'
+'pci_iomap_wc_range'
 'pci_iommu_alloc'
 'pci_iommu_init'
 'pci_ioremap_bar'
 'pci_ioremap_wc_bar'
+'pci_iounmap'
 'pci_iov_add_virtfn'
 'pci_iov_bus_range'
 'pci_iov_init'
@@ -23678,6 +24524,12 @@
 'pci_match_id'
 'pci_max_pasids'
 'pcim_enable_device'
+'pcim_iomap'
+'pcim_iomap_regions'
+'pcim_iomap_regions_request_all'
+'pcim_iomap_table'
+'pcim_iounmap'
+'pcim_iounmap_regions'
 'pci_mmap_fits'
 'pci_mmap_page_range'
 'pci_mmap_resource_range'
@@ -24170,6 +25022,13 @@
 'percpu_alloc_setup'
 'percpu_array_map_lookup_elem'
 'percpu_array_map_seq_show_elem'
+'percpu_counter_add_batch'
+'__percpu_counter_compare'
+'percpu_counter_destroy'
+'__percpu_counter_init'
+'percpu_counter_set'
+'__percpu_counter_sum'
+'percpu_counter_sync'
 'per_cpu_count_show'
 '__percpu_down_read'
 'percpu_down_write'
@@ -24178,6 +25037,15 @@
 '__percpu_init_rwsem'
 'percpu_pagelist_fraction_sysctl_handler'
 'per_cpu_ptr_to_phys'
+'percpu_ref_exit'
+'percpu_ref_init'
+'percpu_ref_is_zero'
+'percpu_ref_kill_and_confirm'
+'percpu_ref_reinit'
+'percpu_ref_resurrect'
+'percpu_ref_switch_to_atomic'
+'percpu_ref_switch_to_atomic_sync'
+'percpu_ref_switch_to_percpu'
 'percpu_rwsem_wait'
 'percpu_rwsem_wake_function'
 'per_cpu_show'
@@ -24187,18 +25055,72 @@
 'perf_aux_output_end'
 'perf_aux_output_flag'
 'perf_aux_output_skip'
+'perf_bp_event'
+'perf_callchain'
 'perf_callchain_kernel'
 'perf_callchain_user'
 'perf_check_microcode'
+'perf_cpu_time_max_percent_handler'
+'perf_event_account_interrupt'
+'perf_event_addr_filters_sync'
+'perf_event_attach_bpf_prog'
+'perf_event_attrs'
+'perf_event_aux_event'
+'perf_event_bpf_event'
+'perf_event_comm'
+'perf_event_create_kernel_counter'
+'perf_event_delayed_put'
+'perf_event_detach_bpf_prog'
+'perf_event_disable'
+'perf_event_disable_inatomic'
+'perf_event_disable_local'
+'perf_event_enable'
+'perf_event_exec'
+'perf_event_exit_cpu'
+'perf_event_exit_task'
 'perf_event_fd_array_get_ptr'
 'perf_event_fd_array_map_free'
 'perf_event_fd_array_put_ptr'
 'perf_event_fd_array_release'
+'perf_event_fork'
+'perf_event_free_task'
+'perf_event_get'
+'perf_event_header__init_id'
+'perf_event_init'
+'perf_event_init_cpu'
+'perf_event_init_task'
+'perf_event_itrace_started'
+'perf_event_ksymbol'
 'perf_event_max_stack_handler'
+'perf_event_mmap'
+'perf_event_namespaces'
 'perf_event_nmi_handler'
+'perf_event_output'
+'perf_event_output_backward'
+'perf_event_output_forward'
+'perf_event__output_id_sample'
+'perf_event_overflow'
+'perf_event_pause'
+'perf_event_period'
 'perf_event_print_debug'
+'perf_event_query_prog_array'
+'perf_event_read_local'
+'perf_event_read_value'
+'perf_event_refresh'
+'perf_event_release_kernel'
 'perf_events_lapic_init'
+'perf_event_sysfs_show'
+'perf_event_task_disable'
+'perf_event_task_enable'
+'__perf_event_task_sched_in'
+'__perf_event_task_sched_out'
+'perf_event_task_tick'
+'perf_event_text_poke'
+'perf_event_update_userpage'
+'perf_event_wakeup'
+'perf_ftrace_event_register'
 'perf_get_aux'
+'perf_get_event'
 'perf_get_regs_user'
 'perf_get_x86_pmu_capability'
 'perf_guest_get_msrs'
@@ -24214,6 +25136,9 @@
 'perf_ibs_stop'
 'perf_ibs_suspend'
 'perf_instruction_pointer'
+'perf_kprobe_destroy'
+'perf_kprobe_init'
+'perf_log_lost_samples'
 'perf_misc_flags'
 'perf_mmap_to_page'
 'perf_msr_probe'
@@ -24225,12 +25150,43 @@
 'perf_output_copy_aux'
 'perf_output_end'
 'perf_output_put_handle'
+'perf_output_sample'
 'perf_output_skip'
 'perf_perm_irq_work_exit'
+'perf_pmu_disable'
+'perf_pmu_enable'
+'perf_pmu_migrate_context'
+'perf_pmu_name'
+'perf_pmu_register'
+'perf_pmu_resched'
+'perf_pmu_snapshot_aux'
+'perf_pmu_unregister'
+'perf_prepare_sample'
+'perf_proc_update_handler'
 'perf_reg_abi'
+'perf_register_guest_info_callbacks'
 'perf_reg_validate'
 'perf_reg_value'
 'perf_restore_debug_store'
+'perf_sample_event_took'
+'perf_sched_cb_dec'
+'perf_sched_cb_inc'
+'___perf_sw_event'
+'__perf_sw_event'
+'perf_swevent_get_recursion_context'
+'perf_swevent_put_recursion_context'
+'perf_swevent_set_period'
+'perf_tp_event'
+'perf_trace_add'
+'perf_trace_buf_alloc'
+'perf_trace_buf_update'
+'perf_trace_del'
+'perf_trace_destroy'
+'perf_trace_init'
+'perf_trace_run_bpf_submit'
+'perf_unregister_guest_info_callbacks'
+'perf_uprobe_destroy'
+'perf_uprobe_init'
 'pericom_do_set_divisor'
 'period_show'
 'period_store'
@@ -24834,6 +25790,9 @@
 'platform_unregister_drivers'
 'play_dead_common'
 'play_idle_precise'
+'plist_add'
+'plist_del'
+'plist_requeue'
 'plt_clk_disable'
 'plt_clk_driver_init'
 'plt_clk_enable'
@@ -25160,6 +26119,7 @@
 'point_resize'
 'poison_show'
 'poke_blanked_console'
+'poke_int3_handler'
 'poking_init'
 'polarity_show'
 'polarity_store'
@@ -25204,6 +26164,7 @@
 'populate_msi_sysfs'
 'populate_pmd'
 'populate_rootfs'
+'__populate_section_memmap'
 'populate_vma_page_range'
 'port_encoding_show'
 'port_number_show'
@@ -25278,6 +26239,7 @@
 'possible_parent_show'
 'possible_parents_open'
 'possible_parents_show'
+'post_alloc_hook'
 'post_init_entity_util_avg'
 'post_set'
 'powercap_init'
@@ -25364,6 +26326,12 @@
 'pps_unregister_cdev'
 'pps_unregister_source'
 'pqdownheap'
+'prandom_bytes'
+'prandom_bytes_state'
+'prandom_seed'
+'prandom_seed_full_state'
+'prandom_u32'
+'prandom_u32_state'
 'prb_commit'
 'prb_fill_curr_block'
 'prb_final_commit'
@@ -25398,12 +26366,15 @@
 'preempt_notifier_inc'
 'preempt_notifier_register'
 'preempt_notifier_unregister'
+'preempt_schedule'
 'preempt_schedule_irq'
+'preempt_schedule_notrace'
 'prefill_possible_map'
 'pre_handler_kretprobe'
 'prepare_creds'
 'prepare_domain_attach_device'
 'prepare_exec_creds'
+'prepare_ftrace_return'
 'prepare_kernel_cred'
 'prepare_keybuf'
 'prepare_kswapd_sleep'
@@ -25442,6 +26413,7 @@
 'print_cfs_stats'
 'print_constraints_debug'
 'print_cpu'
+'print_cpu_info'
 'print_cpu_modalias'
 'print_cpus_isolated'
 'print_cpus_kernel_max'
@@ -25450,9 +26422,13 @@
 'print_daily_error_info'
 'print_dl_rq'
 'print_dl_stats'
+'print_event_filter'
 'print_filtered'
 'print_fixed'
 'print_fixed_last'
+'print_graph_function_flags'
+'print_graph_headers_flags'
+'print_hex_dump'
 'print_ICs'
 'print_IO_APICs'
 'print_ipi_mode'
@@ -25462,6 +26438,8 @@
 'printk_late_init'
 'printk_nmi_direct_enter'
 'printk_nmi_direct_exit'
+'printk_nmi_enter'
+'printk_nmi_exit'
 'printk_percpu_data_ready'
 '__printk_ratelimit'
 '__printk_safe_enter'
@@ -25484,10 +26462,27 @@
 'print_rt_stats'
 'print_section'
 'print_stop_info'
+'print_subsystem_event_filter'
 'print_tainted'
 'print_tickdevice'
+'print_trace_header'
+'print_trace_line'
 'print_tracking'
 'print_trailer'
+'print_type_s16'
+'print_type_s32'
+'print_type_s64'
+'print_type_s8'
+'print_type_string'
+'print_type_symbol'
+'print_type_u16'
+'print_type_u32'
+'print_type_u64'
+'print_type_u8'
+'print_type_x16'
+'print_type_x32'
+'print_type_x64'
+'print_type_x8'
 'print_verifier_state'
 'print_vma_addr'
 'print_worker_info'
@@ -26083,6 +27078,7 @@
 'pstore_dowork'
 'pstore_dump'
 'pstore_evict_inode'
+'pstore_exit_fs'
 'pstore_file_llseek'
 'pstore_file_open'
 'pstore_file_read'
@@ -26215,6 +27211,7 @@
 'ptrace_writedata'
 'pt_read_offset'
 'pt_regs_to_gdb_regs'
+'ptr_to_hashval'
 'pt_show'
 'pts_unix98_lookup'
 'pt_timing_attr_show'
@@ -26358,8 +27355,17 @@
 'put_unused_fd'
 'put_uprobe'
 '__put_user_ns'
+'pvclock_clocksource_read'
+'pvclock_get_pvti_cpu0_va'
 'pvclock_gtod_register_notifier'
 'pvclock_gtod_unregister_notifier'
+'pvclock_read_flags'
+'pvclock_read_wallclock'
+'pvclock_resume'
+'pvclock_set_flags'
+'pvclock_set_pvti_cpu0_va'
+'pvclock_touch_watchdogs'
+'pvclock_tsc_khz'
 'pvh_get_root_pointer'
 'pvm_determine_end_from_reverse'
 'pv_tlb_flush_supported'
@@ -26755,11 +27761,36 @@
 'quota_setquota'
 'quota_setxquota'
 'quota_sync_one'
+'radix_tree_delete'
+'radix_tree_delete_item'
+'radix_tree_gang_lookup'
+'radix_tree_gang_lookup_tag'
+'radix_tree_gang_lookup_tag_slot'
+'radix_tree_init'
+'radix_tree_insert'
+'radix_tree_iter_delete'
+'radix_tree_iter_replace'
+'radix_tree_iter_resume'
+'radix_tree_iter_tag_clear'
+'__radix_tree_lookup'
+'radix_tree_lookup'
+'radix_tree_lookup_slot'
+'radix_tree_maybe_preload'
+'radix_tree_next_chunk'
+'radix_tree_node_rcu_free'
+'radix_tree_preload'
+'__radix_tree_replace'
+'radix_tree_replace_slot'
+'radix_tree_tag_clear'
+'radix_tree_tagged'
+'radix_tree_tag_get'
+'radix_tree_tag_set'
 'raid_disks_show'
 'raid_disks_store'
 'raid_setup'
 'raise_softirq'
 '__raise_softirq_irqoff'
+'raise_softirq_irqoff'
 'ramfs_create'
 'ramfs_fill_super'
 'ramfs_free_fc'
@@ -26795,6 +27826,7 @@
 'ras_debugfs_init'
 'ras_init'
 'ras_userspace_consumers'
+'___ratelimit'
 'rate_limit_us_show'
 'rate_limit_us_store'
 'rational_best_approximation'
@@ -26892,9 +27924,21 @@
 '_raw_write_unlock_irqrestore'
 'rb_alloc'
 'rb_alloc_aux'
+'rb_erase'
+'__rb_erase_color'
+'rb_first'
+'rb_first_postorder'
 'rb_free'
 '__rb_free_aux'
 'rb_free_aux'
+'__rb_insert_augmented'
+'rb_insert_color'
+'rb_last'
+'rb_next'
+'rb_next_postorder'
+'rb_prev'
+'rb_replace_node'
+'rb_replace_node_rcu'
 'rbtree_debugfs_init'
 'rbtree_open'
 'rbtree_show'
@@ -26971,14 +28015,20 @@
 'rcu_init'
 'rcu_init_tasks_generic'
 'rcu_inkernel_boot_has_ended'
+'rcu_irq_enter'
 'rcu_irq_enter_irqson'
+'rcu_irq_exit'
 'rcu_irq_exit_irqson'
 'rcu_irq_exit_preempt'
 'rcu_is_cpu_rrupt_from_idle'
 'rcu_is_idle_cpu'
+'rcu_is_watching'
 'rcu_iw_handler'
 'rcu_jiffies_till_stall_check'
+'rcu_momentary_dyntick_idle'
 'rcu_needs_cpu'
+'rcu_nmi_enter'
+'rcu_nmi_exit'
 'rcu_normal_show'
 'rcu_normal_store'
 'rcu_note_context_switch'
@@ -27105,6 +28155,7 @@
 'rdtgroup_cbm_to_size'
 'rdtgroup_cpus_show'
 'rdtgroup_cpus_write'
+'rdtgroup_exit'
 'rdtgroup_file_write'
 'rdtgroup_init'
 'rdtgroup_kn_lock_live'
@@ -27230,6 +28281,7 @@
 'recalc_sigpending'
 'recalc_sigpending_and_wake'
 'recalc_sigpending_tsk'
+'recalibrate_cpu_khz'
 'receive_fallback_to_copy'
 '__receive_fd'
 'receiver_wake_function'
@@ -27277,7 +28329,13 @@
 'red_zone_show'
 'reenable_swap_slots_cache_unlock'
 'reenter_kprobe'
+'refcount_dec_and_lock'
+'refcount_dec_and_lock_irqsave'
+'refcount_dec_and_mutex_lock'
 'refcount_dec_and_rtnl_lock'
+'refcount_dec_if_one'
+'refcount_dec_not_one'
+'refcount_warn_saturate'
 '__refill_cfs_bandwidth_runtime'
 'refill_obj_stock'
 'refill_stock'
@@ -27343,10 +28401,18 @@
 'register_device_clock'
 'register_die_notifier'
 'register_dock_dependent_device'
+'register_event_command'
 'register_fib_notifier'
 'register_filesystem'
 'register_for_each_vma'
 'register_framebuffer'
+'register_ftrace_command'
+'register_ftrace_direct'
+'register_ftrace_export'
+'__register_ftrace_function'
+'register_ftrace_function'
+'register_ftrace_function_probe'
+'register_ftrace_graph'
 'register_gifconf'
 'register_handler_proc'
 'register_ife_op'
@@ -27399,6 +28465,7 @@
 'register_shrinker'
 'register_shrinker_prepared'
 'register_slot'
+'register_stat_tracer'
 'register_switchdev_blocking_notifier'
 'register_switchdev_notifier'
 'register_syscore_ops'
@@ -27409,7 +28476,10 @@
 'register_sysctl_table'
 'register_sysrq_key'
 'register_tcf_proto_ops'
+'register_trace_event'
 'register_tracepoint_module_notifier'
+'register_tracer'
+'register_trigger_cmds'
 'register_update_efi_random_seed'
 'register_user_hw_breakpoint'
 'register_virtio_device'
@@ -27866,6 +28936,7 @@
 '__report_access'
 'report_access'
 '__report_bad_irq'
+'report_bug'
 'report_error_detected'
 'report_frozen_detected'
 'report_iommu_fault'
@@ -27926,6 +28997,7 @@
 'rescuer_thread'
 'resend_irqs'
 'reserve_bios_regions'
+'reserve_bootmem_region'
 '__reserve_bp_slot'
 'reserve_bp_slot'
 'reserve_ds_buffers'
@@ -27998,6 +29070,7 @@
 'restore_boot_irq_mode'
 'restore_cur'
 'restore_ioapic_entries'
+'restore_processor_state'
 'restore_regs'
 'restore_sigcontext'
 'restrict_link_by_builtin_and_secondary_trusted'
@@ -28060,8 +29133,85 @@
 'rgb_background'
 'rgb_foreground'
 '__rh_alloc'
+'rhashtable_destroy'
+'rhashtable_free_and_destroy'
+'rhashtable_init'
+'rhashtable_insert_slow'
+'rhashtable_walk_enter'
+'rhashtable_walk_exit'
+'rhashtable_walk_next'
+'rhashtable_walk_peek'
+'rhashtable_walk_start_check'
+'rhashtable_walk_stop'
+'rhltable_init'
+'__rht_bucket_nested'
+'rht_bucket_nested'
+'rht_bucket_nested_insert'
 'rh_timer_func'
 'ring3mwait_disable'
+'__ring_buffer_alloc'
+'ring_buffer_alloc_read_page'
+'ring_buffer_bytes_cpu'
+'ring_buffer_change_overwrite'
+'ring_buffer_commit_overrun_cpu'
+'ring_buffer_consume'
+'ring_buffer_discard_commit'
+'ring_buffer_dropped_events_cpu'
+'ring_buffer_empty'
+'ring_buffer_empty_cpu'
+'ring_buffer_entries'
+'ring_buffer_entries_cpu'
+'ring_buffer_event_data'
+'ring_buffer_event_length'
+'ring_buffer_event_time_stamp'
+'ring_buffer_free'
+'ring_buffer_free_read_page'
+'ring_buffer_get'
+'ring_buffer_iter_advance'
+'ring_buffer_iter_dropped'
+'ring_buffer_iter_empty'
+'ring_buffer_iter_peek'
+'ring_buffer_iter_reset'
+'ring_buffer_lock_reserve'
+'ring_buffer_nest_end'
+'ring_buffer_nest_start'
+'ring_buffer_normalize_time_stamp'
+'ring_buffer_nr_dirty_pages'
+'ring_buffer_nr_pages'
+'ring_buffer_oldest_event_ts'
+'ring_buffer_overrun_cpu'
+'ring_buffer_overruns'
+'ring_buffer_peek'
+'ring_buffer_poll_wait'
+'ring_buffer_print_entry_header'
+'ring_buffer_print_page_header'
+'ring_buffer_put'
+'ring_buffer_read_events_cpu'
+'ring_buffer_read_finish'
+'ring_buffer_read_page'
+'ring_buffer_read_prepare'
+'ring_buffer_read_prepare_sync'
+'ring_buffer_read_start'
+'ring_buffer_record_disable'
+'ring_buffer_record_disable_cpu'
+'ring_buffer_record_enable'
+'ring_buffer_record_enable_cpu'
+'ring_buffer_record_is_on'
+'ring_buffer_record_is_set_on'
+'ring_buffer_record_off'
+'ring_buffer_record_on'
+'ring_buffer_reset'
+'ring_buffer_reset_cpu'
+'ring_buffer_reset_online_cpus'
+'ring_buffer_resize'
+'ring_buffer_set_clock'
+'ring_buffer_set_time_stamp_abs'
+'ring_buffer_size'
+'ring_buffer_time_stamp'
+'ring_buffer_time_stamp_abs'
+'ring_buffer_unlock_commit'
+'ring_buffer_wait'
+'ring_buffer_write'
 'ringbuf_map_alloc'
 'ringbuf_map_delete_elem'
 'ringbuf_map_free'
@@ -28261,6 +29411,7 @@
 'rtc_class_open'
 'rtc_cmos_read'
 'rtc_cmos_write'
+'rtc_dev_exit'
 'rtc_dev_fasync'
 'rtc_device_release'
 'rtc_dev_init'
@@ -28472,6 +29623,7 @@
 'run_init_process'
 'run_io_job'
 'run_ksoftirqd'
+'running_clock'
 'run_pages_job'
 'run_posix_cpu_timers'
 'run_rebalance_domains'
@@ -28566,6 +29718,7 @@
 'save_microcode_in_initrd_amd'
 'save_microcode_in_initrd_intel'
 'save_microcode_patch'
+'save_named_trigger'
 'save_processor_state'
 'save_register_state'
 'save_screen'
@@ -28576,6 +29729,26 @@
 'sb_finish_set_opts'
 'sbf_init'
 'sb_init_dio_done_wq'
+'sbitmap_add_wait_queue'
+'sbitmap_any_bit_set'
+'sbitmap_bitmap_show'
+'sbitmap_del_wait_queue'
+'sbitmap_finish_wait'
+'sbitmap_get'
+'sbitmap_get_shallow'
+'sbitmap_init_node'
+'sbitmap_prepare_to_wait'
+'sbitmap_queue_clear'
+'__sbitmap_queue_get'
+'__sbitmap_queue_get_shallow'
+'sbitmap_queue_init_node'
+'sbitmap_queue_min_shallow_depth'
+'sbitmap_queue_resize'
+'sbitmap_queue_show'
+'sbitmap_queue_wake_all'
+'sbitmap_queue_wake_up'
+'sbitmap_resize'
+'sbitmap_show'
 'sb_mark_inode_writeback'
 'sb_min_blocksize'
 'sb_notify_work'
@@ -28617,6 +29790,14 @@
 'sched_cfs_slack_timer'
 'sched_change_group'
 'sched_clear_itmt_support'
+'sched_clock'
+'sched_clock_cpu'
+'sched_clock_idle_sleep_event'
+'sched_clock_idle_wakeup_event'
+'sched_clock_init'
+'sched_clock_stable'
+'sched_clock_tick'
+'sched_clock_tick_stable'
 'sched_cpu_activate'
 'sched_cpu_deactivate'
 'sched_cpu_dying'
@@ -28723,6 +29904,7 @@
 'scmd_printk'
 'scm_fp_dup'
 '__scm_send'
+'scnprintf'
 'scomp_acomp_comp_decomp'
 'scomp_acomp_compress'
 'scomp_acomp_decompress'
@@ -29162,6 +30344,7 @@
 'search_bpf_extables'
 'search_cred_keyrings_rcu'
 'search_exception_tables'
+'search_extable'
 'search_kernel_exception_table'
 'search_module_extables'
 'search_nested_keyrings'
@@ -29872,6 +31055,17 @@
 'sens_index'
 'sens_read'
 'sens_write'
+'seq_buf_bprintf'
+'seq_buf_hex_dump'
+'seq_buf_path'
+'seq_buf_printf'
+'seq_buf_print_seq'
+'seq_buf_putc'
+'seq_buf_putmem'
+'seq_buf_putmem_hex'
+'seq_buf_puts'
+'seq_buf_to_user'
+'seq_buf_vprintf'
 'seq_dentry'
 'seq_escape'
 'seq_escape_mem_ascii'
@@ -29913,6 +31107,7 @@
 'seq_path_root'
 'seq_printf'
 'SEQ_printf'
+'seq_print_ip_sym'
 'seq_putc'
 'seq_put_decimal_ll'
 'seq_put_decimal_ull'
@@ -30200,7 +31395,9 @@
 'set_freezable'
 'set_fs_pwd'
 'set_fs_root'
+'set_ftrace_ops_ro'
 'set_global_limit'
+'set_graph_array'
 'set_groups'
 'set_hardened_usercopy'
 'set_ifa_lifetime'
@@ -30256,6 +31453,7 @@
 'set_mtrr_aps_delayed_init'
 'set_mtrr_ops'
 'set_multi'
+'set_named_trigger_data'
 'set_new_pw'
 'set_next_buddy'
 'set_next_entity'
@@ -30346,8 +31544,10 @@
 'set_tcpmhash_entries'
 'set_thash_entries'
 'set_trace_device'
+'set_tracer_flag'
 'set_translate'
 'set_trigger'
+'set_trigger_filter'
 'set_tsc_mode'
 'set_uhash_entries'
 'setup_acpi_rsdp'
@@ -30366,6 +31566,7 @@
 'setup_boot_APIC_clock'
 'setup_clear_cpu_cap'
 'setup_cpu_entry_areas'
+'setup_cpu_local_masks'
 'setup_data_data_read'
 'setup_data_read'
 'setup_default_ctrlval'
@@ -30435,6 +31636,7 @@
 'setup_v1_file_key_derived'
 'setup_vmw_sched_clock'
 'setup_vq'
+'setup_zone_pageset'
 'set_use_crs'
 'set_user'
 'set_user_nice'
@@ -30512,22 +31714,53 @@
 'sf_setstate'
 'sg_add_device'
 'sg_add_request'
+'__sg_alloc_table'
+'sg_alloc_table'
+'sg_alloc_table_chained'
+'__sg_alloc_table_from_pages'
+'sg_alloc_table_from_pages'
 'sg_build_indirect'
 'sg_build_reserve'
 'sg_complete'
+'sg_copy_buffer'
+'sg_copy_from_buffer'
+'sg_copy_to_buffer'
 'sg_device_destroy'
 'sget'
 'sget_fc'
 'sg_fasync'
 'sg_finish_rem_req'
+'__sg_free_table'
+'sg_free_table'
+'sg_free_table_chained'
 'sg_get_rq_mark'
 'sg_idr_max_id'
+'sg_init_one'
+'sg_init_table'
 'sg_io'
 'sg_ioctl'
 'sgi_partition'
+'sgl_alloc'
+'sgl_alloc_order'
+'sg_last'
+'sgl_free'
+'sgl_free_n_order'
+'sgl_free_order'
+'sg_miter_next'
+'sg_miter_skip'
+'sg_miter_start'
+'sg_miter_stop'
 'sg_mmap'
+'sg_nents'
+'sg_nents_for_len'
 'sg_new_read'
+'sg_next'
 'sg_open'
+'__sg_page_iter_dma_next'
+'__sg_page_iter_next'
+'__sg_page_iter_start'
+'sg_pcopy_from_buffer'
+'sg_pcopy_to_buffer'
 'sg_poll'
 'sg_proc_seq_show_debug'
 'sg_proc_seq_show_dev'
@@ -30550,8 +31783,11 @@
 'sg_scsi_ioctl'
 'sg_vma_fault'
 'sg_write'
+'sg_zero_buffer'
 'sha1_final'
 'sha1_generic_mod_init'
+'sha1_init'
+'sha1_transform'
 'sha224_final'
 'sha224_update'
 'sha256'
@@ -30799,6 +32035,7 @@
 'show_map_vma'
 'show_mark_fhandle'
 'show_max_perf_pct'
+'show_mem'
 'show_min_height'
 'show_min_perf_pct'
 'show_min_width'
@@ -30838,6 +32075,7 @@
 'show_reference_perf'
 '__show_regs'
 'show_regs'
+'show_regs_print_info'
 'show_related_cpus'
 'show_rotate'
 'show_rps_dev_flow_table_cnt'
@@ -30964,8 +32202,10 @@
 'shrink_store'
 'shrink_work'
 'shrink_worker'
+'__shuffle_free_memory'
 'shuffle_pick_tail'
 'shuffle_show'
+'__shuffle_zone'
 'sidtab_cancel_convert'
 'sidtab_context_to_sid'
 'sidtab_convert'
@@ -31053,6 +32293,10 @@
 'simple_set_acl'
 'simple_setattr'
 'simple_statfs'
+'simple_strtol'
+'simple_strtoll'
+'simple_strtoul'
+'simple_strtoull'
 'simple_transaction_get'
 'simple_transaction_read'
 'simple_transaction_release'
@@ -31080,6 +32324,13 @@
 'single_task_running'
 'single_unlink_async'
 'sink_ports_show'
+'siphash_1u32'
+'siphash_1u64'
+'siphash_2u64'
+'siphash_3u32'
+'siphash_3u64'
+'siphash_4u64'
+'__siphash_aligned'
 'sis_cleanup'
 'sis_configure'
 'sis_delayed_enable'
@@ -31322,6 +32573,7 @@
 'sk_get_meminfo'
 'skip_back_repeat_test'
 'skip_prefixes'
+'skip_spaces'
 'skip_tx_en_setup'
 'sk_lookup'
 'sk_lookup_convert_ctx_access'
@@ -31524,6 +32776,7 @@
 'snmp_get_cpu_field'
 'snmp_seq_show'
 'snoop_urb_data'
+'snprintf'
 'sock_addr_convert_ctx_access'
 'sock_addr_func_proto'
 'sock_addr_is_valid_access'
@@ -31751,12 +33004,16 @@
 'software_node_unregister_node_group'
 'software_node_unregister_nodes'
 'software_resume'
+'sort'
+'sort_extable'
 'sort_iommu_table'
 'sort_main_extable'
+'sort_r'
 'sort_range'
 'source_ports_show'
 'space_show'
 'space_store'
+'sparse_buffer_alloc'
 'sparse_index_alloc'
 'sparse_init'
 'sparse_init_nid'
@@ -31914,9 +33171,13 @@
 'split_vma'
 'spp_getpage'
 'sprint_backtrace'
+'sprintf'
+'sprint_oid'
+'sprint_OID'
 '__sprint_symbol'
 'sprint_symbol'
 'sprint_symbol_no_offset'
+'spurious_interrupt'
 'spurious_kernel_fault'
 'sr_audio_ioctl'
 'srbds_parse_cmdline'
@@ -31981,6 +33242,7 @@
 'sr_tray_move'
 'sr_vendor_init'
 'ssb_prctl_set'
+'sscanf'
 'sscanf_key'
 's_show'
 's_start'
@@ -31994,6 +33256,7 @@
 'stable_page_flags'
 'stable_pages_required_show'
 'stable_tree_append'
+'__stack_chk_fail'
 'stack_map_alloc'
 'stack_map_delete_elem'
 'stack_map_free'
@@ -32039,7 +33302,10 @@
 '__start_timer'
 '__start_tty'
 'start_tty'
+'__startup_64'
+'startup_64_setup_env'
 'startup_ioapic_irq'
+'__startup_secondary_64'
 'state_show'
 'state_store'
 'state_synced_show'
@@ -32077,6 +33343,7 @@
 'stop_machine_from_inactive_cpu'
 'stop_machine_park'
 'stop_machine_unpark'
+'stop_machine_yield'
 'stop_merge'
 'stop_nmi'
 'stop_one_cpu'
@@ -32146,15 +33413,28 @@
 'store_user_show'
 'store_virtual'
 'store_wakeup_protocols'
+'stpcpy'
 'str2hashbuf_signed'
 'str2hashbuf_unsigned'
+'strcasecmp'
+'strcat'
+'strchr'
+'strchrnul'
+'strcmp'
+'strcpy'
+'strcspn'
 'stream_open'
 'strict_iomem'
 'strict_strtoul_scaled'
 'strict_work_handler'
+'strim'
+'string_escape_mem'
+'string_escape_mem_ascii'
+'string_get_size'
 'string_to_av_perm'
 'string_to_context_struct'
 'string_to_security_class'
+'string_unescape'
 'stripe_ctr'
 'stripe_dtr'
 'stripe_end_io'
@@ -32164,11 +33444,25 @@
 'stripe_map_range_sector'
 'stripe_map_sector'
 'stripe_status'
+'strlcat'
+'strlcpy'
+'strlen'
+'strncasecmp'
+'strncat'
+'strnchr'
+'strnchrnul'
+'strncmp'
+'strncpy'
 'strncpy_from_kernel_nofault'
+'strncpy_from_user'
 'strncpy_from_user_nofault'
 'strndup_user'
+'strnlen'
+'strnlen_user'
 'strnlen_user_nofault'
+'strnstr'
 'strp_abort_strp'
+'strpbrk'
 'strp_check_rcv'
 'strp_data_ready'
 'strp_dev_init'
@@ -32185,12 +33479,19 @@
 '__strp_unpause'
 'strp_unpause'
 'strp_work'
+'strrchr'
 'str_read'
+'strreplace'
+'strscpy'
+'strscpy_pad'
+'strsep'
 'strset_cleanup_data'
 'strset_fill_reply'
 'strset_parse_request'
 'strset_prepare_data'
 'strset_reply_size'
+'strspn'
+'strstr'
 'str_to_user'
 'subcaches_show'
 'subcaches_store'
@@ -32379,6 +33680,8 @@
 'switch_mm'
 'switch_mm_irqs_off'
 'switch_task_namespaces'
+'__switch_to'
+'switch_to_new_gdt'
 'switch_to_sld'
 '__switch_to_xtra'
 'swnode_register'
@@ -32485,6 +33788,7 @@
 'sync_page_io'
 'sync_rcu_exp_select_cpus'
 'sync_rcu_exp_select_node_cpus'
+'sync_regs'
 'sync_speed_show'
 'sync_state_only_show'
 'sync_state_resume_initcall'
@@ -32494,9 +33798,13 @@
 '__sys_accept4'
 '__sys_accept4_file'
 '__sys_bind'
+'syscall_enter_from_user_mode'
+'syscall_enter_from_user_mode_prepare'
 'syscall_enter_from_user_mode_work'
+'syscall_exit_to_user_mode'
 'syscall_exit_to_user_mode_work'
 'syscall_exit_work'
+'syscall_init'
 'syscall_regfunc'
 'syscall_unregfunc'
 'syscall_user_dispatch'
@@ -32569,6 +33877,7 @@
 'sysfs_kf_write'
 'sysfs_kill_sb'
 'sysfs_link_change_owner'
+'__sysfs_match_string'
 'sysfs_merge_group'
 'sysfs_move_dir_ns'
 'sysfs_notify'
@@ -32591,6 +33900,7 @@
 'sysfs_slab_alias'
 'sysfs_slab_release'
 'sysfs_slab_unlink'
+'sysfs_streq'
 'sysfs_unbind_tick_dev'
 'sysfs_unbreak_active_protection'
 'sysfs_unmerge_group'
@@ -32603,6 +33913,7 @@
 '__sys_listen'
 'syslog_print'
 'syslog_print_all'
+'sys_ni_syscall'
 '__sys_recvfrom'
 '__sys_recvmmsg'
 '____sys_recvmsg'
@@ -32669,19 +33980,36 @@
 'system_root_device_release'
 'system_trusted_keyring_init'
 '__sysvec_apic_timer_interrupt'
+'sysvec_apic_timer_interrupt'
 '__sysvec_call_function'
+'sysvec_call_function'
 '__sysvec_call_function_single'
+'sysvec_call_function_single'
 '__sysvec_deferred_error'
+'sysvec_deferred_error'
 '__sysvec_error_interrupt'
+'sysvec_error_interrupt'
 '__sysvec_irq_move_cleanup'
+'sysvec_irq_move_cleanup'
 '__sysvec_irq_work'
+'sysvec_irq_work'
 '__sysvec_kvm_asyncpf_interrupt'
+'sysvec_kvm_asyncpf_interrupt'
+'sysvec_kvm_posted_intr_ipi'
+'sysvec_kvm_posted_intr_nested_ipi'
 '__sysvec_kvm_posted_intr_wakeup_ipi'
+'sysvec_kvm_posted_intr_wakeup_ipi'
 '__sysvec_reboot'
+'sysvec_reboot'
+'sysvec_reschedule_ipi'
 '__sysvec_spurious_apic_interrupt'
+'sysvec_spurious_apic_interrupt'
 '__sysvec_thermal'
+'sysvec_thermal'
 '__sysvec_threshold'
+'sysvec_threshold'
 '__sysvec_x86_platform_ipi'
+'sysvec_x86_platform_ipi'
 'sysvipc_find_ipc'
 'sysvipc_msg_proc_show'
 'sysvipc_proc_next'
@@ -32760,6 +34088,8 @@
 'task_cls_state'
 'task_contending'
 'task_cputime_adjusted'
+'task_curr'
+'task_current_syscall'
 'task_dead_fair'
 'task_dump_owner'
 'task_file_seq_get_next'
@@ -33543,6 +34873,11 @@
 'text_poke_loc_init'
 'text_poke_queue'
 'text_poke_sync'
+'textsearch_destroy'
+'textsearch_find_continuous'
+'textsearch_prepare'
+'textsearch_register'
+'textsearch_unregister'
 'tfa_get_event_constraints'
 'tfilter_notify'
 'tg_bps_limit'
@@ -33648,6 +34983,7 @@
 'thinking_detect'
 'thinkpad_e530_quirk'
 'thin_provisioning_show'
+'__this_cpu_preempt_check'
 'this_tty'
 'thp_get_unmapped_area'
 'thread_cpu_clock_get'
@@ -33824,6 +35160,9 @@
 'timer_list_start'
 'timer_list_stop'
 'timer_migration_handler'
+'timerqueue_add'
+'timerqueue_del'
+'timerqueue_iterate_next'
 'timer_reduce'
 'timers_dead_cpu'
 'timer_show'
@@ -33910,10 +35249,13 @@
 'total_trans_show'
 'touch_all_softlockup_watchdogs'
 'touch_atime'
+'touch_buffer'
 'touchscreen_parse_properties'
 'touchscreen_report_pos'
 'touchscreen_set_mt_pos'
 'touchscreen_set_params'
+'touch_softlockup_watchdog'
+'touch_softlockup_watchdog_sched'
 'touch_softlockup_watchdog_sync'
 'to_utf8'
 'tpacket_destruct_skb'
@@ -33978,6 +35320,7 @@
 'tpm_common_write'
 'tpm_default_chip'
 'tpm_dev_async_work'
+'tpm_dev_common_exit'
 'tpm_dev_common_init'
 'tpm_dev_release'
 'tpm_devs_release'
@@ -34047,7 +35390,65 @@
 'tps68470_gpio_probe'
 'tps68470_gpio_set'
 'tps68470_probe'
+'trace_add_event_call'
+'trace_array_destroy'
+'trace_array_find'
+'trace_array_find_get'
+'trace_array_get'
+'trace_array_get_by_name'
+'trace_array_init_printk'
+'trace_array_printk'
+'trace_array_printk_buf'
+'trace_array_put'
+'trace_array_set_clr_event'
+'trace_array_vprintk'
+'__trace_bprintk'
+'__trace_bputs'
+'trace_buffered_event_disable'
+'trace_buffered_event_enable'
+'trace_buffer_lock_reserve'
+'trace_buffer_unlock_commit_nostack'
+'trace_buffer_unlock_commit_regs'
+'trace_call_bpf'
+'trace_clock'
+'trace_clock_counter'
+'trace_clock_global'
+'trace_clock_in_ns'
+'trace_clock_jiffies'
+'trace_clock_local'
+'trace_clock_x86_tsc'
+'trace_create_file'
+'trace_default_header'
+'trace_define_field'
+'trace_dump_stack'
+'__trace_early_add_events'
+'trace_empty'
+'trace_event_buffer_commit'
+'trace_event_buffer_lock_reserve'
+'trace_event_buffer_reserve'
+'trace_event_enable_cmd_record'
+'trace_event_enable_disable'
+'trace_event_enable_tgid_record'
+'trace_event_eval_update'
+'trace_event_follow_fork'
+'trace_event_get_offsets'
+'trace_event_ignore_this_pid'
+'trace_event_init'
+'trace_event_raw_init'
+'trace_event_read_lock'
+'trace_event_read_unlock'
+'trace_event_reg'
+'trace_event_trigger_enable_disable'
 'trace_fill_super'
+'trace_filter_add_remove_task'
+'trace_find_cmdline'
+'trace_find_event_field'
+'trace_find_filtered_pid'
+'trace_find_mark'
+'trace_find_next_entry'
+'trace_find_next_entry_inc'
+'trace_find_tgid'
+'trace_free_pid_list'
 'tracefs_create_dir'
 'tracefs_create_file'
 'tracefs_create_instance_dir'
@@ -34060,10 +35461,22 @@
 'tracefs_show_options'
 'tracefs_syscall_mkdir'
 'tracefs_syscall_rmdir'
+'trace_function'
+'trace_get_event_file'
+'trace_get_user'
+'__trace_graph_entry'
+'trace_graph_entry'
+'trace_graph_function'
+'__trace_graph_return'
+'trace_graph_return'
+'trace_handle_return'
+'trace_ignore_this_task'
+'trace_init'
 'trace_initcall_finish_cb'
 'trace_initcall_start_cb'
 'trace_init_flags_sys_enter'
 'trace_init_flags_sys_exit'
+'trace_init_global_iter'
 'trace_init_perf_perm_irq_work_exit'
 '__traceiter_add_device_randomness'
 '__traceiter_add_device_to_group'
@@ -34106,6 +35519,7 @@
 '__traceiter_bounce_map_single'
 '__traceiter_bounce_unmap_single'
 '__traceiter_bpf_test_finish'
+'__traceiter_bpf_trace_printk'
 '__traceiter_break_lease_block'
 '__traceiter_break_lease_noblock'
 '__traceiter_break_lease_unblock'
@@ -34146,14 +35560,20 @@
 '__traceiter_clk_set_rate_range'
 '__traceiter_clk_unprepare'
 '__traceiter_clk_unprepare_complete'
+'__traceiter_clock_disable'
+'__traceiter_clock_enable'
+'__traceiter_clock_set_rate'
 '__traceiter_cma_alloc'
 '__traceiter_cma_release'
 '__traceiter_compact_retry'
 '__traceiter_console'
 '__traceiter_consume_skb'
+'__traceiter_cpu_frequency'
+'__traceiter_cpu_frequency_limits'
 '__traceiter_cpuhp_enter'
 '__traceiter_cpuhp_exit'
 '__traceiter_cpuhp_multi_enter'
+'__traceiter_cpu_idle'
 '__traceiter_credit_entropy_bits'
 '__traceiter_dax_insert_mapping'
 '__traceiter_dax_insert_pfn_mkwrite'
@@ -34175,6 +35595,11 @@
 '__traceiter_detach_device_from_domain'
 '__traceiter_devfreq_frequency'
 '__traceiter_devfreq_monitor'
+'__traceiter_device_pm_callback_end'
+'__traceiter_device_pm_callback_start'
+'__traceiter_dev_pm_qos_add_request'
+'__traceiter_dev_pm_qos_remove_request'
+'__traceiter_dev_pm_qos_update_request'
 '__traceiter_dma_fence_destroy'
 '__traceiter_dma_fence_emit'
 '__traceiter_dma_fence_enable_signal'
@@ -34520,11 +35945,19 @@
 '__traceiter_percpu_create_chunk'
 '__traceiter_percpu_destroy_chunk'
 '__traceiter_percpu_free_percpu'
+'__traceiter_pm_qos_add_request'
+'__traceiter_pm_qos_remove_request'
+'__traceiter_pm_qos_update_flags'
+'__traceiter_pm_qos_update_request'
+'__traceiter_pm_qos_update_target'
 '__traceiter_posix_lock_inode'
+'__traceiter_power_domain_target'
+'__traceiter_powernv_throttle'
 '__traceiter_prandom_u32'
 '__traceiter_pseudo_lock_l2'
 '__traceiter_pseudo_lock_l3'
 '__traceiter_pseudo_lock_mem_latency'
+'__traceiter_pstate_sample'
 '__traceiter_push_to_pool'
 '__traceiter_pwm_apply'
 '__traceiter_pwm_get'
@@ -34566,6 +35999,11 @@
 '__traceiter_remove_device_from_group'
 '__traceiter_reschedule_entry'
 '__traceiter_reschedule_exit'
+'__traceiter_rpm_idle'
+'__traceiter_rpm_resume'
+'__traceiter_rpm_return_int'
+'__traceiter_rpm_suspend'
+'__traceiter_rpm_usage'
 '__traceiter_rseq_ip_fixup'
 '__traceiter_rseq_update'
 '__traceiter_rss_stat'
@@ -34643,6 +36081,7 @@
 '__traceiter_spurious_apic_entry'
 '__traceiter_spurious_apic_exit'
 '__traceiter_start_task_reaping'
+'__traceiter_suspend_resume'
 '__traceiter_swiotlb_bounced'
 '__traceiter_sys_enter'
 '__traceiter_sys_exit'
@@ -34691,6 +36130,8 @@
 '__traceiter_vm_unmapped_area'
 '__traceiter_wait_on_page_writeback'
 '__traceiter_wake_reaper'
+'__traceiter_wakeup_source_activate'
+'__traceiter_wakeup_source_deactivate'
 '__traceiter_wbc_writepage'
 '__traceiter_wbt_lat'
 '__traceiter_wbt_stat'
@@ -34800,20 +36241,142 @@
 '__traceiter_xhci_urb_dequeue'
 '__traceiter_xhci_urb_enqueue'
 '__traceiter_xhci_urb_giveback'
+'trace_keep_overwrite'
+'trace_kprobe_error_injectable'
+'trace_kprobe_on_func_entry'
+'trace_latency_header'
 'trace_mmap_lock_reg'
 'trace_mmap_lock_unreg'
 'trace_module_has_bad_taint'
 'trace_mount'
+'trace_nop_print'
 'trace_open'
+'trace_output_call'
 'trace_pagefault_reg'
 'trace_pagefault_unreg'
+'trace_parser_get_init'
+'trace_parser_put'
+'trace_parse_run_command'
+'trace_pid_next'
+'trace_pid_show'
+'trace_pid_start'
+'trace_pid_write'
 'tracepoint_add_func'
 'tracepoint_module_notify'
+'tracepoint_printk_sysctl'
 'tracepoint_probe_register'
 'tracepoint_probe_register_prio'
 'tracepoint_probe_unregister'
+'trace_print_array_seq'
+'trace_print_bitmask_seq'
+'trace_print_bprintk_msg_only'
+'trace_print_bputs_msg_only'
+'trace_print_context'
+'trace_print_flags_seq'
+'trace_print_graph_duration'
+'trace_print_hex_dump_seq'
+'trace_print_hex_seq'
+'__trace_printk'
+'trace_printk_control'
+'trace_printk_init_buffers'
+'trace_printk_seq'
+'trace_printk_start_comm'
+'trace_print_lat_context'
+'trace_print_lat_fmt'
+'trace_print_printk_msg_only'
+'trace_print_seq'
+'trace_print_symbols_seq'
+'trace_probe_add_file'
+'trace_probe_append'
+'trace_probe_cleanup'
+'trace_probe_compare_arg_type'
+'traceprobe_define_arg_fields'
+'traceprobe_free_probe_arg'
+'trace_probe_get_file_link'
+'trace_probe_init'
+'trace_probe_log_clear'
+'__trace_probe_log_err'
+'trace_probe_log_init'
+'trace_probe_log_set_index'
+'trace_probe_match_command_args'
+'traceprobe_parse_event_name'
+'traceprobe_parse_probe_arg'
+'trace_probe_register_event_call'
+'trace_probe_remove_file'
+'traceprobe_set_print_fmt'
+'traceprobe_split_symbol_offset'
+'trace_probe_unlink'
+'traceprobe_update_arg'
+'trace_put_event_file'
+'__trace_puts'
+'trace_raw_output_prep'
+'trace_rb_cpu_prepare'
 'trace_release'
+'trace_remove_event_call'
+'tracer_init'
+'tracer_tracing_is_on'
+'tracer_tracing_off'
+'tracer_tracing_on'
+'trace_run_command'
+'trace_seq_bitmask'
+'trace_seq_bprintf'
+'trace_seq_hex_dump'
+'trace_seq_path'
+'trace_seq_printf'
+'trace_seq_print_sym'
+'trace_seq_putc'
+'trace_seq_putmem'
+'trace_seq_putmem_hex'
+'trace_seq_puts'
+'trace_seq_to_user'
+'trace_seq_vprintf'
+'trace_set_clr_event'
+'trace_set_options'
 'trace_show'
+'__trace_stack'
+'trace_total_entries'
+'trace_total_entries_cpu'
+'trace_vbprintk'
+'trace_vprintk'
+'tracing_alloc_snapshot'
+'tracing_check_open_get_tr'
+'tracing_cond_snapshot_data'
+'tracing_generic_entry_update'
+'tracing_init_dentry'
+'tracing_is_disabled'
+'tracing_is_enabled'
+'tracing_is_on'
+'tracing_iter_reset'
+'tracing_log_err'
+'tracing_lseek'
+'tracing_off'
+'tracing_on'
+'tracing_open_generic'
+'tracing_open_generic_tr'
+'tracing_prog_func_proto'
+'tracing_record_cmdline'
+'tracing_record_taskinfo'
+'tracing_record_taskinfo_sched_switch'
+'tracing_record_tgid'
+'tracing_reset_all_online_cpus'
+'tracing_reset_online_cpus'
+'tracing_resize_ring_buffer'
+'tracing_set_clock'
+'tracing_set_cpumask'
+'tracing_set_time_stamp_abs'
+'tracing_set_tracer'
+'tracing_snapshot'
+'tracing_snapshot_alloc'
+'tracing_snapshot_cond'
+'tracing_snapshot_cond_disable'
+'tracing_snapshot_cond_enable'
+'tracing_start'
+'tracing_start_cmdline_record'
+'tracing_start_tgid_record'
+'tracing_stop'
+'tracing_stop_cmdline_record'
+'tracing_stop_tgid_record'
+'tracing_update_buffers'
 '__track_dentry_update'
 '__track_inode'
 'track_pfn_copy'
@@ -34869,9 +36432,12 @@
 'trie_get_next_key'
 'trie_lookup_elem'
 'trie_update_elem'
+'trigger_data_free'
 'trigger_event'
 'trigger_load_balance'
+'trigger_process_regex'
 'trim_dev_dax_range'
+'trim_init_extable'
 'trim_marked'
 'trim_stale_devices'
 'trip_point_hyst_show'
@@ -34925,6 +36491,10 @@
 'try_to_wake_up'
 'try_to_writeback_inodes_sb'
 'try_wait_for_completion'
+'tsc_early_init'
+'tsc_init'
+'tsc_restore_sched_clock_state'
+'tsc_save_sched_clock_state'
 'tsc_show'
 'tsc_store_and_check_tsc_adjust'
 'tsc_verify_tsc_adjust'
@@ -35311,6 +36881,12 @@
 'ubb_show'
 'ubb_store'
 'uc_decode_notifier'
+'ucs2_as_utf8'
+'ucs2_strlen'
+'ucs2_strncmp'
+'ucs2_strnlen'
+'ucs2_strsize'
+'ucs2_utf8size'
 'ucs_cmp'
 'ud_bmove'
 'ud_clear'
@@ -35636,6 +37212,9 @@
 'unlock_system_sleep'
 'unlock_two_nondirectories'
 'unlock_vector_lock'
+'unlz4'
+'unlzma'
+'unlzo'
 '__unmap_hugepage_range'
 'unmap_hugepage_range'
 '__unmap_hugepage_range_final'
@@ -35659,6 +37238,7 @@
 'unmerge_ksm_pages'
 'unoptimize_kprobe'
 'unpack_to_rootfs'
+'unpause_named_trigger'
 'unpin_user_page'
 'unpin_user_pages'
 'unpin_user_pages_dirty_lock'
@@ -35684,10 +37264,18 @@
 'unregister_dev_dax'
 'unregister_die_notifier'
 '__unregister_dummy'
+'unregister_event_command'
 'unregister_fair_sched_group'
 'unregister_fib_notifier'
 'unregister_filesystem'
 'unregister_framebuffer'
+'unregister_ftrace_command'
+'unregister_ftrace_direct'
+'unregister_ftrace_export'
+'__unregister_ftrace_function'
+'unregister_ftrace_function'
+'unregister_ftrace_function_probe_func'
+'unregister_ftrace_graph'
 'unregister_handler_proc'
 'unregister_hw_breakpoint'
 'unregister_ife_op'
@@ -35734,12 +37322,15 @@
 'unregister_sched_domain_sysctl'
 'unregister_shrinker'
 'unregister_snapshot'
+'unregister_stat_tracer'
 'unregister_switchdev_blocking_notifier'
 'unregister_switchdev_notifier'
 'unregister_syscore_ops'
 'unregister_sysctl_table'
 'unregister_sysrq_key'
 'unregister_tcf_proto_ops'
+'__unregister_trace_event'
+'unregister_trace_event'
 'unregister_tracepoint_module_notifier'
 'unregister_virtio_device'
 'unregister_virtio_driver'
@@ -35756,6 +37347,7 @@
 'unshare_nsproxy_namespaces'
 'unshare_userns'
 'unsolicited_report_interval'
+'unsynchronized_tsc'
 'unthrottle_cfs_rq'
 'untrack_pfn'
 'untrack_pfn_moved'
@@ -35767,6 +37359,8 @@
 'unwind_next_frame'
 '__unwind_start'
 'unxlate_dev_mem_ptr'
+'unxz'
+'unzstd'
 'up'
 'update_and_free_page'
 'update_attr'
@@ -35780,6 +37374,7 @@
 'update_children'
 'update_classid_sock'
 'update_closid_rmid'
+'update_cond_flag'
 'update_cpu_closid_rmid'
 'update_cpumasks_hier'
 'update_curr'
@@ -35801,6 +37396,7 @@
 'update_extent_range'
 'update_fast_timekeeper'
 'update_flag'
+'update_function_graph_func'
 'update_group_capacity'
 '__update_idle_core'
 'update_if_frozen'
@@ -36319,6 +37915,7 @@
 'userns_put'
 'user_page_pipe_buf_try_steal'
 'user_path_at_empty'
+'user_path_create'
 'user_preparse'
 'user_read'
 'user_reader_timeout'
@@ -36339,6 +37936,7 @@
 'use_zero_pages_store'
 'use_zero_page_store'
 'using_compacted_format'
+'using_native_sched_clock'
 'usleep_range'
 'utf16s_to_utf8s'
 'utf32_to_utf8'
@@ -36351,6 +37949,9 @@
 'utsns_owner'
 'utsns_put'
 'uts_proc_notify'
+'uuid_gen'
+'uuid_is_valid'
+'uuid_parse'
 'uuid_show'
 'v2_check_quota_file'
 'v2_free_file_info'
@@ -36392,6 +37993,7 @@
 'var_to_display'
 '__var_waitqueue'
 'var_wake_function'
+'vbin_printf'
 'vc_allocate'
 'vc_cons_allocated'
 'vc_deallocate'
@@ -36754,6 +38356,17 @@
 'vmcoreinfo_show'
 'vmcore_init'
 'vmemdup_user'
+'vmemmap_alloc_block'
+'vmemmap_alloc_block_buf'
+'vmemmap_p4d_populate'
+'vmemmap_pgd_populate'
+'vmemmap_pmd_populate'
+'vmemmap_populate'
+'vmemmap_populate_basepages'
+'vmemmap_populate_print_last'
+'vmemmap_pte_populate'
+'vmemmap_pud_populate'
+'vmemmap_verify'
 '__vm_enough_memory'
 'vm_events_fold_cpu'
 'vmf_insert_mixed'
@@ -36862,8 +38475,12 @@
 '__vring_new_virtqueue'
 'vring_new_virtqueue'
 'vring_transport_features'
+'vscnprintf'
 'vsmp_apic_post_init'
 'vsmp_init'
+'vsnprintf'
+'vsprintf'
+'vsscanf'
 'vsyscall_setup'
 'vt8237_force_enable_hpet'
 'vt_clr_kbd_mode_bit'
@@ -37051,6 +38668,7 @@
 '_warn_unseeded_randomness'
 'warn_unsupported'
 'watchdog_core_data_release'
+'watchdog_dev_exit'
 'watchdog_dev_init'
 'watchdog_dev_register'
 'watchdog_dev_unregister'
@@ -37150,6 +38768,7 @@
 'whole_disk_show'
 'width_show'
 'will_become_orphaned_pgrp'
+'within_error_injection_list'
 'within_kprobe_blacklist'
 'wMaxPacketSize_show'
 'woken_wake_function'
@@ -37515,6 +39134,7 @@
 '__x64_sys_pciconfig_iobase'
 '__x64_sys_pciconfig_read'
 '__x64_sys_pciconfig_write'
+'__x64_sys_perf_event_open'
 '__x64_sys_personality'
 '__x64_sys_pidfd_getfd'
 '__x64_sys_pidfd_open'
@@ -37708,6 +39328,8 @@
 '__x64_sys_waitpid'
 '__x64_sys_write'
 '__x64_sys_writev'
+'x86_64_start_kernel'
+'x86_64_start_reservations'
 'x86_acpi_enter_sleep_state'
 'x86_acpi_suspend_lowlevel'
 'x86_add_exclusive'
@@ -37784,6 +39406,7 @@
 'x86_pmu_swap_task_ctx'
 'x86_pnpbios_disabled'
 'x86_rdrand_setup'
+'x86_read_arch_cap_msr'
 'x86_release_hardware'
 'x86_report_nx'
 'x86_reserve_hardware'
@@ -37800,7 +39423,44 @@
 'x86_vector_select'
 'x86_virt_spec_ctrl'
 'x86_wallclock_init'
+'__xa_alloc'
+'__xa_alloc_cyclic'
 'xacct_add_tsk'
+'__xa_clear_mark'
+'xa_clear_mark'
+'__xa_cmpxchg'
+'xa_delete_node'
+'xa_destroy'
+'__xa_erase'
+'xa_erase'
+'xa_extract'
+'xa_find'
+'xa_find_after'
+'xa_get_mark'
+'xa_get_order'
+'__xa_insert'
+'xa_load'
+'xas_clear_mark'
+'xas_create_range'
+'__xa_set_mark'
+'xa_set_mark'
+'xas_find'
+'xas_find_conflict'
+'xas_find_marked'
+'xas_get_mark'
+'xas_init_marks'
+'xas_load'
+'__xas_next'
+'xas_nomem'
+'xas_pause'
+'__xas_prev'
+'xas_set_mark'
+'xas_split'
+'xas_split_alloc'
+'xas_store'
+'__xa_store'
+'xa_store'
+'xa_store_range'
 '__xattr_check_inode'
 'xattr_find_entry'
 'xattr_full_name'
@@ -37808,7 +39468,21 @@
 'xattr_permission'
 'xattr_resolve_name'
 'xattr_supported_namespace'
+'xbc_debug_dump'
+'xbc_destroy_all'
+'xbc_init'
 'xbc_make_cmdline'
+'xbc_node_compose_key_after'
+'xbc_node_find_child'
+'xbc_node_find_next_key_value'
+'xbc_node_find_next_leaf'
+'xbc_node_find_value'
+'xbc_node_get_child'
+'xbc_node_get_data'
+'xbc_node_get_next'
+'xbc_node_get_parent'
+'xbc_node_index'
+'xbc_root_node'
 'xbc_snprint_cmdline'
 'xdbc_free_ring'
 'xdbc_get_page'
@@ -38206,6 +39880,7 @@
 'xhci_debugfs_init'
 'xhci_debugfs_regset'
 'xhci_debugfs_remove_endpoint'
+'xhci_debugfs_remove_root'
 'xhci_debugfs_remove_slot'
 'xhci_device_name_show'
 'xhci_disable_slot'
@@ -38501,6 +40176,16 @@
 'xt_unregister_table'
 'xt_unregister_target'
 'xt_unregister_targets'
+'xxh32'
+'xxh32_copy_state'
+'xxh32_digest'
+'xxh32_reset'
+'xxh32_update'
+'xxh64'
+'xxh64_copy_state'
+'xxh64_digest'
+'xxh64_reset'
+'xxh64_update'
 'xz_dec_bcj_create'
 'xz_dec_bcj_reset'
 'xz_dec_bcj_run'
@@ -38614,6 +40299,7 @@
 'zone_pcp_disable'
 'zone_pcp_enable'
 'zone_pcp_reset'
+'zone_pcp_update'
 'zone_reclaimable_pages'
 'zone_set_pageset_high_and_batch'
 'zone_sizes_init'

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Wed, Feb 10, 2021 at 11:11 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:
> > On Wed, Feb 10, 2021 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
> > >
> > > On Tue, Feb 09, 2021 at 02:00:29PM -0800, Andrii Nakryiko wrote:
> > >
> > > SNIP
> > >
> > > > > > > I'm still trying to build the kernel.. however ;-)
> > > > > > >
> > > > > > > patch below adds the ftrace check only for static functions
> > > > > > > and lets the externa go through.. but as you said, in this
> > > > > > > case we'll need to figure out the 'notrace' and other checks
> > > > > > > ftrace is doing
> > > > > > >
> > > > > > > jirka
> > > > > > >
> > > > > > >
> > > > > > > ---
> > > > > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > > > > index b124ec20a689..4d147406cfa5 100644
> > > > > > > --- a/btf_encoder.c
> > > > > > > +++ b/btf_encoder.c
> > > > > > > @@ -734,7 +734,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> > > > > > >                         continue;
> > > > > > >                 if (!has_arg_names(cu, &fn->proto))
> > > > > > >                         continue;
> > > > > > > -               if (functions_cnt) {
> > > > > > > +               if (!fn->external && functions_cnt) {
> > > > > >
> > > > > > I wouldn't trust DWARF, honestly. Wouldn't checking GLOBAL vs LOCAL
> > > > > > FUNC ELF symbol be more reliable?
> > > > >
> > > > > that'd mean extra bsearch on each processed function,
> > > > > on the ther hand, we'are already slow ;-) I'll check
> > > > > how big the slowdown would be
> > > > >
> > > >
> > > > We currently record addresses and do binary search. Now we need to
> > > > record address + size and still do binary search with a slightly
> > > > different semantics (find closest entry >= addr). Then just check that
> > > > it overlaps, taking into account the length of the function code. It
> > > > shouldn't result in a noticeable slowdown. Might be actually faster,
> > > > because we might avoid callback function call costs.
> > >
> > > I'm still not sure how to handle the external check for function via elf,
> >
> > I might be missing something, but don't all functions have
> > corresponding ELF symbols? And then symbol can have LOCAL or GLOBAL
> > type. LOCALs are supposed to be not visible outside respective CUs (so
> > correspond to static functions), while GLOBALs are extern-able funcs.
> > So if func's symbol is GLOBAL, it should be ok to assume it's
> > attachable (not inlined, at least).
>
> sure I know where the info is, I was just hesitating to add another bsearch
>
> anyway, I checked the BTF data before and after this change (the dwarf check above)
> and it looks like there are many global functions that are not attachable
>
> attaching diff of BTF functions below - before (.old) and after (.new) adding
> all external functions.. all added (+) functions are external and do not have
> ftrace address - I verified some of them in disassembly and they are not attachable
>
> I think we should check ftrace addresses also for global functions
>
> I'm of course assuming BTF should have only attachable functions


Oh, ok, I didn't realize so many functions are not attachable. Yeah,
let's not do anything special for GLOBALs, given we are fixing the
ftrace addr search anyway.

>
> jirka
>
>
> ---
> --- /tmp/pahole.test.PbD/funcs-vmlinux-x86-gcc-1.old    2021-02-10 19:51:44.727865204 +0100
> +++ /tmp/pahole.test.PbD/funcs-vmlinux-x86-gcc-1.new    2021-02-10 19:51:44.836865827 +0100
> @@ -1662,6 +1662,7 @@

[...]

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:

SNIP

> > but below is change for checking that ftrace addrs are within elf functions
> >
> > seems to work in my tests, I'll run some more tests and send full patch
> 
> It seems unnecessarily convoluted. I was thinking about something like
> this (the diff will totally be screwed up by gmail, and I haven't even
> compiled it):
> 
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..8162b238bd43 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
> **paddrs, __u64 *pcount)
>         return 0;
>  }
> 
> +struct func_seg { __u64 start; __u64 end; };
> +
> +static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
> +{
> +       size_t l = 0, r = len - 1, m;
> +
> +       while (l < r) {
> +               m = l + (r - l + 1) / 2;
> +               if (segs[m].start <= addr)
> +                       l = m;
> +               else
> +                       r = m - 1;
> +       }
> +
> +       return segs[l].start <= addr && addr < segs[l].end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
>         __u64 *addrs, count, i;
> @@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
> struct funcs_layout *fl)
>                 __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> 
>                 /* Make sure function is within ftrace addresses. */
> -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> +               if (func_exists(addrs, count, addr))

you pass addrs in here, but you mean func_seg array
filled with elf functions start/end values, right?

>                         /*
>                          * We iterate over sorted array, so we can easily skip
>                          * not valid item and move following valid field into
> 
> 
> So the idea is to use address segments and check whether there is a
> segment that overlaps with a given address by first binary searching
> for a segment with the largest starting address that is <= addr. And
> then just confirming that segment does overlap with the requested
> address.
> 
> WDYT?

I liked the approach below because it got rid of that bsearch ;-)

but yea, yours seems to be less code chage and staighforward

jirka

> 
> >
> > jirka
> >
> >
> > ---
> > diff --git a/btf_encoder.c b/btf_encoder.c
> > index b124ec20a689..548a12847f99 100644
> > --- a/btf_encoder.c
> > +++ b/btf_encoder.c
> > @@ -36,6 +36,7 @@ struct funcs_layout {
> >  struct elf_function {
> >         const char      *name;
> >         unsigned long    addr;
> > +       unsigned long    end;
> >         unsigned long    sh_addr;
> >         bool             generated;
> >  };
> > @@ -44,7 +45,7 @@ static struct elf_function *functions;
> >  static int functions_alloc;
> >  static int functions_cnt;
> >
> > -static int functions_cmp(const void *_a, const void *_b)
> > +static int functions_cmp_name(const void *_a, const void *_b)
> >  {
> >         const struct elf_function *a = _a;
> >         const struct elf_function *b = _b;
> > @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
> >         return strcmp(a->name, b->name);
> >  }
> >
> > +static int functions_cmp_addr(const void *_a, const void *_b)
> > +{
> > +       const struct elf_function *a = _a;
> > +       const struct elf_function *b = _b;
> > +
> > +       if (a->addr == b->addr)
> > +               return 0;
> > +       return a->addr < b->addr ? -1 : 1;
> > +}
> > +
> >  static void delete_functions(void)
> >  {
> >         free(functions);
> > @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
> >
> >         functions[functions_cnt].name = name;
> >         functions[functions_cnt].addr = elf_sym__value(sym);
> > +       functions[functions_cnt].end = (__u64) -1;
> >         functions[functions_cnt].sh_addr = sh.sh_addr;
> >         functions[functions_cnt].generated = false;
> >         functions_cnt++;
> > @@ -236,9 +248,25 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
> >         return 0;
> >  }
> >
> > +static bool is_addr_in_func(__u64 addr, struct elf_function *func, bool kmod)
> > +{
> > +       /*
> > +        * For vmlinux image both addrs[x] and functions[x]::addr
> > +        * values are final address and are comparable.
> > +        *
> > +        * For kernel module addrs[x] is final address, but
> > +        * functions[x]::addr is relative address within section
> > +        * and needs to be relocated by adding sh_addr.
> > +        */
> > +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> > +       __u64 end = kmod ? func->end+ func->sh_addr : func->end;
> > +
> > +       return start <= addr && addr < end;
> > +}
> > +
> >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >  {
> > -       __u64 *addrs, count, i;
> > +       __u64 *addrs, count, i_func, i_addr;
> >         int functions_valid = 0;
> >         bool kmod = false;
> >
> > @@ -266,43 +294,62 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >                 return 0;
> >         }
> >
> > -       qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> > -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> > -
> >         /*
> > -        * Let's got through all collected functions and filter
> > -        * out those that are not in ftrace.
> > +        * Sort both functions and addrs so we can iterate
> > +        * both of them simultaneously and found matching
> > +        * func/addr pairs.
> >          */
> > -       for (i = 0; i < functions_cnt; i++) {
> > -               struct elf_function *func = &functions[i];
> > -               /*
> > -                * For vmlinux image both addrs[x] and functions[x]::addr
> > -                * values are final address and are comparable.
> > -                *
> > -                * For kernel module addrs[x] is final address, but
> > -                * functions[x]::addr is relative address within section
> > -                * and needs to be relocated by adding sh_addr.
> > -                */
> > -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > +       qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> > +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
> > +
> > +       for (i_func = 0, i_addr = 0; i_func < functions_cnt; i_func++) {
> > +               struct elf_function *func = &functions[i_func];
> > +
> > +               if (i_func + 1 < functions_cnt)
> > +                       func->end = functions[i_func + 1].addr;
> > +
> > +               for (; i_addr < count; i_addr++) {
> > +                       __u64 addr = addrs[i_addr];
> > +
> > +                       /* Functions are  ahead, catch up with addrs. */
> > +                       if (addr < func->addr)
> > +                               continue;
> > +
> > +                       /* Addr is within function - mark function as valid. */
> > +                       if (is_addr_in_func(addr, func, kmod)) {
> > +                               /*
> > +                                * We iterate over sorted array, so we can easily skip
> > +                                * not valid item and move following valid field into
> > +                                * its place, and still keep the 'new' array sorted.
> > +                                */
> > +                               if (i_func != functions_valid)
> > +                                       functions[functions_valid] = functions[i_func];
> > +                               functions_valid++;
> > +                               i_addr++;
> > +                       }
> >
> > -               /* Make sure function is within ftrace addresses. */
> > -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> >                         /*
> > -                        * We iterate over sorted array, so we can easily skip
> > -                        * not valid item and move following valid field into
> > -                        * its place, and still keep the 'new' array sorted.
> > +                        * Addrs are ahead, catch up with functions, or we just
> > +                        * found valid function and want to move to another.
> >                          */
> > -                       if (i != functions_valid)
> > -                               functions[functions_valid] = functions[i];
> > -                       functions_valid++;
> > +                       break;
> >                 }
> >         }
> >
> > +       if (btf_elf__verbose) {
> > +               printf("Found %d functions out of %d symbols and %llu ftrace addresses.\n",
> > +                       functions_valid, functions_cnt, count);
> > +       }
> > +
> >         functions_cnt = functions_valid;
> >         free(addrs);
> >
> > -       if (btf_elf__verbose)
> > -               printf("Found %d functions!\n", functions_cnt);
> > +       /*
> > +        * And finaly sort 'valid' functions by name,
> > +        * so find_function can be used.
> > +        */
> > +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
> > +
> >         return 0;
> >  }
> >
> > @@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
> >         struct elf_function key = { .name = name };
> >
> >         return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> > -                      functions_cmp);
> > +                      functions_cmp_name);
> >  }
> >
> >  static bool btf_name_char_ok(char c, bool first)
> >
> 

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Wed, Feb 10, 2021 at 09:13:47PM +0100, Jiri Olsa wrote:
> On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:
> 
> SNIP
> 
> > > but below is change for checking that ftrace addrs are within elf functions
> > >
> > > seems to work in my tests, I'll run some more tests and send full patch
> > 
> > It seems unnecessarily convoluted. I was thinking about something like
> > this (the diff will totally be screwed up by gmail, and I haven't even
> > compiled it):
> > 
> > diff --git a/btf_encoder.c b/btf_encoder.c
> > index b124ec20a689..8162b238bd43 100644
> > --- a/btf_encoder.c
> > +++ b/btf_encoder.c
> > @@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
> > **paddrs, __u64 *pcount)
> >         return 0;
> >  }
> > 
> > +struct func_seg { __u64 start; __u64 end; };
> > +
> > +static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
> > +{
> > +       size_t l = 0, r = len - 1, m;
> > +
> > +       while (l < r) {
> > +               m = l + (r - l + 1) / 2;
> > +               if (segs[m].start <= addr)
> > +                       l = m;
> > +               else
> > +                       r = m - 1;
> > +       }
> > +
> > +       return segs[l].start <= addr && addr < segs[l].end;
> > +}
> > +
> >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >  {
> >         __u64 *addrs, count, i;
> > @@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
> > struct funcs_layout *fl)
> >                 __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > 
> >                 /* Make sure function is within ftrace addresses. */
> > -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> > +               if (func_exists(addrs, count, addr))
> 
> you pass addrs in here, but you mean func_seg array
> filled with elf functions start/end values, right?
> 
> >                         /*
> >                          * We iterate over sorted array, so we can easily skip
> >                          * not valid item and move following valid field into
> > 
> > 
> > So the idea is to use address segments and check whether there is a
> > segment that overlaps with a given address by first binary searching
> > for a segment with the largest starting address that is <= addr. And
> > then just confirming that segment does overlap with the requested
> > address.
> > 
> > WDYT?

heya,
with your approach I ended up with change below, it gives me same
results as with the previous change

I think I'll separate the kmod bool address computation later on,
but I did not want to confuse this change for now

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index b124ec20a689..34df08f2fb4e 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -36,6 +36,7 @@ struct funcs_layout {
 struct elf_function {
 	const char	*name;
 	unsigned long	 addr;
+	unsigned long	 end;
 	unsigned long	 sh_addr;
 	bool		 generated;
 };
@@ -44,7 +45,7 @@ static struct elf_function *functions;
 static int functions_alloc;
 static int functions_cnt;
 
-static int functions_cmp(const void *_a, const void *_b)
+static int functions_cmp_name(const void *_a, const void *_b)
 {
 	const struct elf_function *a = _a;
 	const struct elf_function *b = _b;
@@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
 	return strcmp(a->name, b->name);
 }
 
+static int functions_cmp_addr(const void *_a, const void *_b)
+{
+	const struct elf_function *a = _a;
+	const struct elf_function *b = _b;
+
+	if (a->addr == b->addr)
+		return 0;
+	return a->addr < b->addr ? -1 : 1;
+}
+
 static void delete_functions(void)
 {
 	free(functions);
@@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
 
 	functions[functions_cnt].name = name;
 	functions[functions_cnt].addr = elf_sym__value(sym);
+	functions[functions_cnt].end = (__u64) -1;
 	functions[functions_cnt].sh_addr = sh.sh_addr;
 	functions[functions_cnt].generated = false;
 	functions_cnt++;
@@ -236,6 +248,40 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
 	return 0;
 }
 
+static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
+			  __u64 count, bool kmod)
+{
+	/*
+	 * For vmlinux image both addrs[x] and functions[x]::addr
+	 * values are final address and are comparable.
+	 *
+	 * For kernel module addrs[x] is final address, but
+	 * functions[x]::addr is relative address within section
+	 * and needs to be relocated by adding sh_addr.
+	 */
+	__u64 start = kmod ? func->addr + func->sh_addr : func->addr;
+	__u64 end   = kmod ? func->end + func->sh_addr : func->end;
+
+	size_t l = 0, r = count - 1, m;
+	__u64 addr = 0;
+
+	while (l < r) {
+		m = l + (r - l + 1) / 2;
+		addr = addrs[m];
+
+		if (start <= addr && addr < end)
+			return true;
+
+		if (start <= addr)
+			r = m - 1;
+		else
+			l = m;
+	}
+
+	addr = addrs[l];
+	return start <= addr && addr < end;
+}
+
 static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 {
 	__u64 *addrs, count, i;
@@ -267,7 +313,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 	}
 
 	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
-	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
+	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
 
 	/*
 	 * Let's got through all collected functions and filter
@@ -275,18 +321,12 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 	 */
 	for (i = 0; i < functions_cnt; i++) {
 		struct elf_function *func = &functions[i];
-		/*
-		 * For vmlinux image both addrs[x] and functions[x]::addr
-		 * values are final address and are comparable.
-		 *
-		 * For kernel module addrs[x] is final address, but
-		 * functions[x]::addr is relative address within section
-		 * and needs to be relocated by adding sh_addr.
-		 */
-		__u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
+
+		if (i + 1 < functions_cnt)
+			func->end = functions[i + 1].addr;
 
 		/* Make sure function is within ftrace addresses. */
-		if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
+		if (is_ftrace_func(func, addrs, count, kmod)) {
 			/*
 			 * We iterate over sorted array, so we can easily skip
 			 * not valid item and move following valid field into
@@ -303,6 +343,8 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 
 	if (btf_elf__verbose)
 		printf("Found %d functions!\n", functions_cnt);
+
+	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
 	return 0;
 }
 
@@ -312,7 +354,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
 	struct elf_function key = { .name = name };
 
 	return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
-		       functions_cmp);
+		       functions_cmp_name);
 }
 
 static bool btf_name_char_ok(char c, bool first)

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Sedat Dilek]

On Thu, Feb 11, 2021 at 4:08 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Wed, Feb 10, 2021 at 09:13:47PM +0100, Jiri Olsa wrote:
> > On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:
> >
> > SNIP
> >
> > > > but below is change for checking that ftrace addrs are within elf functions
> > > >
> > > > seems to work in my tests, I'll run some more tests and send full patch
> > >
> > > It seems unnecessarily convoluted. I was thinking about something like
> > > this (the diff will totally be screwed up by gmail, and I haven't even
> > > compiled it):
> > >
> > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > index b124ec20a689..8162b238bd43 100644
> > > --- a/btf_encoder.c
> > > +++ b/btf_encoder.c
> > > @@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
> > > **paddrs, __u64 *pcount)
> > >         return 0;
> > >  }
> > >
> > > +struct func_seg { __u64 start; __u64 end; };
> > > +
> > > +static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
> > > +{
> > > +       size_t l = 0, r = len - 1, m;
> > > +
> > > +       while (l < r) {
> > > +               m = l + (r - l + 1) / 2;
> > > +               if (segs[m].start <= addr)
> > > +                       l = m;
> > > +               else
> > > +                       r = m - 1;
> > > +       }
> > > +
> > > +       return segs[l].start <= addr && addr < segs[l].end;
> > > +}
> > > +
> > >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> > >  {
> > >         __u64 *addrs, count, i;
> > > @@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
> > > struct funcs_layout *fl)
> > >                 __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > >
> > >                 /* Make sure function is within ftrace addresses. */
> > > -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> > > +               if (func_exists(addrs, count, addr))
> >
> > you pass addrs in here, but you mean func_seg array
> > filled with elf functions start/end values, right?
> >
> > >                         /*
> > >                          * We iterate over sorted array, so we can easily skip
> > >                          * not valid item and move following valid field into
> > >
> > >
> > > So the idea is to use address segments and check whether there is a
> > > segment that overlaps with a given address by first binary searching
> > > for a segment with the largest starting address that is <= addr. And
> > > then just confirming that segment does overlap with the requested
> > > address.
> > >
> > > WDYT?
>
> heya,
> with your approach I ended up with change below, it gives me same
> results as with the previous change
>
> I think I'll separate the kmod bool address computation later on,
> but I did not want to confuse this change for now
>

I have applied your diff on top of pahole-v1.20 with Yonghong Son's
"btf_encoder: sanitize non-regular int base type" applied.
This is on x86-64 with LLVM-12, so I am not directly affected.
If it is out of interest I can offer vmlinux (or .*btf* files) w/ and
w/o your diff.

- Sedat -

> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..34df08f2fb4e 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -36,6 +36,7 @@ struct funcs_layout {
>  struct elf_function {
>         const char      *name;
>         unsigned long    addr;
> +       unsigned long    end;
>         unsigned long    sh_addr;
>         bool             generated;
>  };
> @@ -44,7 +45,7 @@ static struct elf_function *functions;
>  static int functions_alloc;
>  static int functions_cnt;
>
> -static int functions_cmp(const void *_a, const void *_b)
> +static int functions_cmp_name(const void *_a, const void *_b)
>  {
>         const struct elf_function *a = _a;
>         const struct elf_function *b = _b;
> @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
>         return strcmp(a->name, b->name);
>  }
>
> +static int functions_cmp_addr(const void *_a, const void *_b)
> +{
> +       const struct elf_function *a = _a;
> +       const struct elf_function *b = _b;
> +
> +       if (a->addr == b->addr)
> +               return 0;
> +       return a->addr < b->addr ? -1 : 1;
> +}
> +
>  static void delete_functions(void)
>  {
>         free(functions);
> @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
>
>         functions[functions_cnt].name = name;
>         functions[functions_cnt].addr = elf_sym__value(sym);
> +       functions[functions_cnt].end = (__u64) -1;
>         functions[functions_cnt].sh_addr = sh.sh_addr;
>         functions[functions_cnt].generated = false;
>         functions_cnt++;
> @@ -236,6 +248,40 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
>         return 0;
>  }
>
> +static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
> +                         __u64 count, bool kmod)
> +{
> +       /*
> +        * For vmlinux image both addrs[x] and functions[x]::addr
> +        * values are final address and are comparable.
> +        *
> +        * For kernel module addrs[x] is final address, but
> +        * functions[x]::addr is relative address within section
> +        * and needs to be relocated by adding sh_addr.
> +        */
> +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> +       __u64 end   = kmod ? func->end + func->sh_addr : func->end;
> +
> +       size_t l = 0, r = count - 1, m;
> +       __u64 addr = 0;
> +
> +       while (l < r) {
> +               m = l + (r - l + 1) / 2;
> +               addr = addrs[m];
> +
> +               if (start <= addr && addr < end)
> +                       return true;
> +
> +               if (start <= addr)
> +                       r = m - 1;
> +               else
> +                       l = m;
> +       }
> +
> +       addr = addrs[l];
> +       return start <= addr && addr < end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
>         __u64 *addrs, count, i;
> @@ -267,7 +313,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>         }
>
>         qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
>
>         /*
>          * Let's got through all collected functions and filter
> @@ -275,18 +321,12 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>          */
>         for (i = 0; i < functions_cnt; i++) {
>                 struct elf_function *func = &functions[i];
> -               /*
> -                * For vmlinux image both addrs[x] and functions[x]::addr
> -                * values are final address and are comparable.
> -                *
> -                * For kernel module addrs[x] is final address, but
> -                * functions[x]::addr is relative address within section
> -                * and needs to be relocated by adding sh_addr.
> -                */
> -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> +
> +               if (i + 1 < functions_cnt)
> +                       func->end = functions[i + 1].addr;
>
>                 /* Make sure function is within ftrace addresses. */
> -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> +               if (is_ftrace_func(func, addrs, count, kmod)) {
>                         /*
>                          * We iterate over sorted array, so we can easily skip
>                          * not valid item and move following valid field into
> @@ -303,6 +343,8 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>
>         if (btf_elf__verbose)
>                 printf("Found %d functions!\n", functions_cnt);
> +
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
>         return 0;
>  }
>
> @@ -312,7 +354,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
>         struct elf_function key = { .name = name };
>
>         return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> -                      functions_cmp);
> +                      functions_cmp_name);
>  }
>
>  static bool btf_name_char_ok(char c, bool first)
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/YCVIWzq0quDQm6bn%40krava.

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Thu, Feb 11, 2021 at 04:43:48PM +0100, Sedat Dilek wrote:

SNIP

> > > filled with elf functions start/end values, right?
> > >
> > > >                         /*
> > > >                          * We iterate over sorted array, so we can easily skip
> > > >                          * not valid item and move following valid field into
> > > >
> > > >
> > > > So the idea is to use address segments and check whether there is a
> > > > segment that overlaps with a given address by first binary searching
> > > > for a segment with the largest starting address that is <= addr. And
> > > > then just confirming that segment does overlap with the requested
> > > > address.
> > > >
> > > > WDYT?
> >
> > heya,
> > with your approach I ended up with change below, it gives me same
> > results as with the previous change
> >
> > I think I'll separate the kmod bool address computation later on,
> > but I did not want to confuse this change for now
> >
> 
> I have applied your diff on top of pahole-v1.20 with Yonghong Son's
> "btf_encoder: sanitize non-regular int base type" applied.
> This is on x86-64 with LLVM-12, so I am not directly affected.
> If it is out of interest I can offer vmlinux (or .*btf* files) w/ and
> w/o your diff.

if you could run your tests/workloads and check the new change does not
break your stuff, that'd be great

we need soem testsuite ;-) I have some stupid test script which runs over
few vmlinux binaries and check the diff in BTF data.. problem is that these
vmlinux binaries are ~300M each, so it's not great for sharing

also I was checking if we could use BPF_BTF_LOAD syscall and load BTF in
kernel and back at the end of pahole processing to check it's valid ;-)

thanks,
jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Sedat Dilek]

On Thu, Feb 11, 2021 at 5:07 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Thu, Feb 11, 2021 at 04:43:48PM +0100, Sedat Dilek wrote:
>
> SNIP
>
> > > > filled with elf functions start/end values, right?
> > > >
> > > > >                         /*
> > > > >                          * We iterate over sorted array, so we can easily skip
> > > > >                          * not valid item and move following valid field into
> > > > >
> > > > >
> > > > > So the idea is to use address segments and check whether there is a
> > > > > segment that overlaps with a given address by first binary searching
> > > > > for a segment with the largest starting address that is <= addr. And
> > > > > then just confirming that segment does overlap with the requested
> > > > > address.
> > > > >
> > > > > WDYT?
> > >
> > > heya,
> > > with your approach I ended up with change below, it gives me same
> > > results as with the previous change
> > >
> > > I think I'll separate the kmod bool address computation later on,
> > > but I did not want to confuse this change for now
> > >
> >
> > I have applied your diff on top of pahole-v1.20 with Yonghong Son's
> > "btf_encoder: sanitize non-regular int base type" applied.
> > This is on x86-64 with LLVM-12, so I am not directly affected.
> > If it is out of interest I can offer vmlinux (or .*btf* files) w/ and
> > w/o your diff.
>
> if you could run your tests/workloads and check the new change does not
> break your stuff, that'd be great
>
> we need soem testsuite ;-) I have some stupid test script which runs over
> few vmlinux binaries and check the diff in BTF data.. problem is that these
> vmlinux binaries are ~300M each, so it's not great for sharing
>
> also I was checking if we could use BPF_BTF_LOAD syscall and load BTF in
> kernel and back at the end of pahole processing to check it's valid ;-)
>

Just finished a new build.

What I did:

cd /path/to/linux/git
rm -v .*btf* vmlinux*
<re-run my build-script with modified pahole>

I collected some commands in CBL issue #1297.

$ /usr/sbin/bpftool btf dump file vmlinux | rg 'vfs_truncate|bpf_d_path'
[22259] TYPEDEF 'btf_bpf_d_path' type_id=22260
[29970] FUNC 'vfs_truncate' type_id=29969 linkage=static

Tests?

$MAKE $MAKE_OPTS -C tools/testing/selftests/bpf/

^^^ ???

- Sedat -

[1] https://github.com/ClangBuiltLinux/linux/issues/1297

> thanks,
> jirka
>

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Nathan Chancellor]

On Thu, Feb 11, 2021 at 04:08:11PM +0100, Jiri Olsa wrote:
> On Wed, Feb 10, 2021 at 09:13:47PM +0100, Jiri Olsa wrote:
> > On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:
> > 
> > SNIP
> > 
> > > > but below is change for checking that ftrace addrs are within elf functions
> > > >
> > > > seems to work in my tests, I'll run some more tests and send full patch
> > > 
> > > It seems unnecessarily convoluted. I was thinking about something like
> > > this (the diff will totally be screwed up by gmail, and I haven't even
> > > compiled it):
> > > 
> > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > index b124ec20a689..8162b238bd43 100644
> > > --- a/btf_encoder.c
> > > +++ b/btf_encoder.c
> > > @@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
> > > **paddrs, __u64 *pcount)
> > >         return 0;
> > >  }
> > > 
> > > +struct func_seg { __u64 start; __u64 end; };
> > > +
> > > +static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
> > > +{
> > > +       size_t l = 0, r = len - 1, m;
> > > +
> > > +       while (l < r) {
> > > +               m = l + (r - l + 1) / 2;
> > > +               if (segs[m].start <= addr)
> > > +                       l = m;
> > > +               else
> > > +                       r = m - 1;
> > > +       }
> > > +
> > > +       return segs[l].start <= addr && addr < segs[l].end;
> > > +}
> > > +
> > >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> > >  {
> > >         __u64 *addrs, count, i;
> > > @@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
> > > struct funcs_layout *fl)
> > >                 __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > > 
> > >                 /* Make sure function is within ftrace addresses. */
> > > -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> > > +               if (func_exists(addrs, count, addr))
> > 
> > you pass addrs in here, but you mean func_seg array
> > filled with elf functions start/end values, right?
> > 
> > >                         /*
> > >                          * We iterate over sorted array, so we can easily skip
> > >                          * not valid item and move following valid field into
> > > 
> > > 
> > > So the idea is to use address segments and check whether there is a
> > > segment that overlaps with a given address by first binary searching
> > > for a segment with the largest starting address that is <= addr. And
> > > then just confirming that segment does overlap with the requested
> > > address.
> > > 
> > > WDYT?
> 
> heya,
> with your approach I ended up with change below, it gives me same
> results as with the previous change
> 
> I think I'll separate the kmod bool address computation later on,
> but I did not want to confuse this change for now
> 
> jirka

This still looks good against the original and reduced problematic
configurations for aarch64.

Tested-by: Nathan Chancellor <nathan@kernel.org>

> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..34df08f2fb4e 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -36,6 +36,7 @@ struct funcs_layout {
>  struct elf_function {
>  	const char	*name;
>  	unsigned long	 addr;
> +	unsigned long	 end;
>  	unsigned long	 sh_addr;
>  	bool		 generated;
>  };
> @@ -44,7 +45,7 @@ static struct elf_function *functions;
>  static int functions_alloc;
>  static int functions_cnt;
>  
> -static int functions_cmp(const void *_a, const void *_b)
> +static int functions_cmp_name(const void *_a, const void *_b)
>  {
>  	const struct elf_function *a = _a;
>  	const struct elf_function *b = _b;
> @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
>  	return strcmp(a->name, b->name);
>  }
>  
> +static int functions_cmp_addr(const void *_a, const void *_b)
> +{
> +	const struct elf_function *a = _a;
> +	const struct elf_function *b = _b;
> +
> +	if (a->addr == b->addr)
> +		return 0;
> +	return a->addr < b->addr ? -1 : 1;
> +}
> +
>  static void delete_functions(void)
>  {
>  	free(functions);
> @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
>  
>  	functions[functions_cnt].name = name;
>  	functions[functions_cnt].addr = elf_sym__value(sym);
> +	functions[functions_cnt].end = (__u64) -1;
>  	functions[functions_cnt].sh_addr = sh.sh_addr;
>  	functions[functions_cnt].generated = false;
>  	functions_cnt++;
> @@ -236,6 +248,40 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
>  	return 0;
>  }
>  
> +static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
> +			  __u64 count, bool kmod)
> +{
> +	/*
> +	 * For vmlinux image both addrs[x] and functions[x]::addr
> +	 * values are final address and are comparable.
> +	 *
> +	 * For kernel module addrs[x] is final address, but
> +	 * functions[x]::addr is relative address within section
> +	 * and needs to be relocated by adding sh_addr.
> +	 */
> +	__u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> +	__u64 end   = kmod ? func->end + func->sh_addr : func->end;
> +
> +	size_t l = 0, r = count - 1, m;
> +	__u64 addr = 0;
> +
> +	while (l < r) {
> +		m = l + (r - l + 1) / 2;
> +		addr = addrs[m];
> +
> +		if (start <= addr && addr < end)
> +			return true;
> +
> +		if (start <= addr)
> +			r = m - 1;
> +		else
> +			l = m;
> +	}
> +
> +	addr = addrs[l];
> +	return start <= addr && addr < end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
>  	__u64 *addrs, count, i;
> @@ -267,7 +313,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  	}
>  
>  	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> -	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> +	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
>  
>  	/*
>  	 * Let's got through all collected functions and filter
> @@ -275,18 +321,12 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  	 */
>  	for (i = 0; i < functions_cnt; i++) {
>  		struct elf_function *func = &functions[i];
> -		/*
> -		 * For vmlinux image both addrs[x] and functions[x]::addr
> -		 * values are final address and are comparable.
> -		 *
> -		 * For kernel module addrs[x] is final address, but
> -		 * functions[x]::addr is relative address within section
> -		 * and needs to be relocated by adding sh_addr.
> -		 */
> -		__u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> +
> +		if (i + 1 < functions_cnt)
> +			func->end = functions[i + 1].addr;
>  
>  		/* Make sure function is within ftrace addresses. */
> -		if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> +		if (is_ftrace_func(func, addrs, count, kmod)) {
>  			/*
>  			 * We iterate over sorted array, so we can easily skip
>  			 * not valid item and move following valid field into
> @@ -303,6 +343,8 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  
>  	if (btf_elf__verbose)
>  		printf("Found %d functions!\n", functions_cnt);
> +
> +	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
>  	return 0;
>  }
>  
> @@ -312,7 +354,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
>  	struct elf_function key = { .name = name };
>  
>  	return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> -		       functions_cmp);
> +		       functions_cmp_name);
>  }
>  
>  static bool btf_name_char_ok(char c, bool first)
> 

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Thu, Feb 11, 2021 at 7:08 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Wed, Feb 10, 2021 at 09:13:47PM +0100, Jiri Olsa wrote:
> > On Wed, Feb 10, 2021 at 10:20:20AM -0800, Andrii Nakryiko wrote:
> >
> > SNIP
> >
> > > > but below is change for checking that ftrace addrs are within elf functions
> > > >
> > > > seems to work in my tests, I'll run some more tests and send full patch
> > >
> > > It seems unnecessarily convoluted. I was thinking about something like
> > > this (the diff will totally be screwed up by gmail, and I haven't even
> > > compiled it):
> > >
> > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > index b124ec20a689..8162b238bd43 100644
> > > --- a/btf_encoder.c
> > > +++ b/btf_encoder.c
> > > @@ -236,6 +236,23 @@ get_kmod_addrs(struct btf_elf *btfe, __u64
> > > **paddrs, __u64 *pcount)
> > >         return 0;
> > >  }
> > >
> > > +struct func_seg { __u64 start; __u64 end; };
> > > +
> > > +static int func_exists(struct func_seg *segs, size_t len, __u64 addr)
> > > +{
> > > +       size_t l = 0, r = len - 1, m;
> > > +
> > > +       while (l < r) {
> > > +               m = l + (r - l + 1) / 2;
> > > +               if (segs[m].start <= addr)
> > > +                       l = m;
> > > +               else
> > > +                       r = m - 1;
> > > +       }
> > > +
> > > +       return segs[l].start <= addr && addr < segs[l].end;
> > > +}
> > > +
> > >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> > >  {
> > >         __u64 *addrs, count, i;
> > > @@ -286,7 +303,7 @@ static int setup_functions(struct btf_elf *btfe,
> > > struct funcs_layout *fl)
> > >                 __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > >
> > >                 /* Make sure function is within ftrace addresses. */
> > > -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> > > +               if (func_exists(addrs, count, addr))
> >
> > you pass addrs in here, but you mean func_seg array
> > filled with elf functions start/end values, right?
> >
> > >                         /*
> > >                          * We iterate over sorted array, so we can easily skip
> > >                          * not valid item and move following valid field into
> > >
> > >
> > > So the idea is to use address segments and check whether there is a
> > > segment that overlaps with a given address by first binary searching
> > > for a segment with the largest starting address that is <= addr. And
> > > then just confirming that segment does overlap with the requested
> > > address.
> > >
> > > WDYT?
>
> heya,
> with your approach I ended up with change below, it gives me same
> results as with the previous change
>
> I think I'll separate the kmod bool address computation later on,
> but I did not want to confuse this change for now
>
> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..34df08f2fb4e 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -36,6 +36,7 @@ struct funcs_layout {
>  struct elf_function {
>         const char      *name;
>         unsigned long    addr;
> +       unsigned long    end;
>         unsigned long    sh_addr;
>         bool             generated;
>  };
> @@ -44,7 +45,7 @@ static struct elf_function *functions;
>  static int functions_alloc;
>  static int functions_cnt;
>
> -static int functions_cmp(const void *_a, const void *_b)
> +static int functions_cmp_name(const void *_a, const void *_b)
>  {
>         const struct elf_function *a = _a;
>         const struct elf_function *b = _b;
> @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
>         return strcmp(a->name, b->name);
>  }
>
> +static int functions_cmp_addr(const void *_a, const void *_b)
> +{
> +       const struct elf_function *a = _a;
> +       const struct elf_function *b = _b;
> +
> +       if (a->addr == b->addr)
> +               return 0;
> +       return a->addr < b->addr ? -1 : 1;
> +}
> +
>  static void delete_functions(void)
>  {
>         free(functions);
> @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
>
>         functions[functions_cnt].name = name;
>         functions[functions_cnt].addr = elf_sym__value(sym);
> +       functions[functions_cnt].end = (__u64) -1;
>         functions[functions_cnt].sh_addr = sh.sh_addr;
>         functions[functions_cnt].generated = false;
>         functions_cnt++;
> @@ -236,6 +248,40 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
>         return 0;
>  }
>
> +static int is_ftrace_func(struct elf_function *func, __u64 *addrs,

return bool, not int?

> +                         __u64 count, bool kmod)
> +{
> +       /*
> +        * For vmlinux image both addrs[x] and functions[x]::addr
> +        * values are final address and are comparable.
> +        *
> +        * For kernel module addrs[x] is final address, but
> +        * functions[x]::addr is relative address within section
> +        * and needs to be relocated by adding sh_addr.
> +        */
> +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> +       __u64 end   = kmod ? func->end + func->sh_addr : func->end;
> +
> +       size_t l = 0, r = count - 1, m;
> +       __u64 addr = 0;
> +
> +       while (l < r) {
> +               m = l + (r - l + 1) / 2;
> +               addr = addrs[m];
> +
> +               if (start <= addr && addr < end)
> +                       return true;

this extra check on each step shouldn't be necessary

> +
> +               if (start <= addr)

I don't think this is correct, start == addr is actually a good case,
but you'll do r = m - 1, skipping it. See below about invariants.

> +                       r = m - 1;
> +               else
> +                       l = m;

So in my previous example I assumed we have address ranges for ftrace
section, which is exactly the opposite from what we have. So this
binary search should be a bit different. start <= addr seems wrong
here as well.

The invariant here should be that addr[r] is the smallest address that
is >= than function start addr, right? Except the corner case where
there is no such r, but for that we have a final check in the return
below. If you wanted to use index l, you'd need to change the
invariant to find the largest addr, such that it is < end, but that
seems a bit convoluted.

So, with that, I think it should be like this:

size_t l = 0, r = count - 1, m;

/* make sure we don't use invalid r */
if (count == 0) return false;

while (l < r) {
    /* note no +1 in this case, it's so that at the end, when you
     * have, say, l = 0, and r = 1, you try l first, not r.
     * Otherwise you might end in in the infinite loop when r never == l.
     */
    m = l + (r - l) / 2;
    addr = addrs[m];

    if (addr >= start)
        /* we satisfy invariant, so tighten r */
        r = m;
    else
        /* m is not good enough as l, maybe m + 1 will be */
        l = m + 1;
}

return start <= addrs[r] && addrs[r] < end;


So, basically, r is maintained as a valid index always, while we
constantly try to tighten the l.

Does this make sense?


> +       }
> +
> +       addr = addrs[l];
> +       return start <= addr && addr < end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
>         __u64 *addrs, count, i;
> @@ -267,7 +313,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>         }
>
>         qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);

See below assumptions about function end. If we get it from ELF, you
don't need to do this extra sort, right?

>
>         /*
>          * Let's got through all collected functions and filter
> @@ -275,18 +321,12 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>          */
>         for (i = 0; i < functions_cnt; i++) {
>                 struct elf_function *func = &functions[i];
> -               /*
> -                * For vmlinux image both addrs[x] and functions[x]::addr
> -                * values are final address and are comparable.
> -                *
> -                * For kernel module addrs[x] is final address, but
> -                * functions[x]::addr is relative address within section
> -                * and needs to be relocated by adding sh_addr.
> -                */
> -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> +
> +               if (i + 1 < functions_cnt)
> +                       func->end = functions[i + 1].addr;

This makes a bunch of unnecessary assumptions about functions layout.
But why, if we have STT_FUNC symbol with function size, so that we
know the function end right when we collect function info.

>
>                 /* Make sure function is within ftrace addresses. */
> -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> +               if (is_ftrace_func(func, addrs, count, kmod)) {
>                         /*
>                          * We iterate over sorted array, so we can easily skip
>                          * not valid item and move following valid field into
> @@ -303,6 +343,8 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>
>         if (btf_elf__verbose)
>                 printf("Found %d functions!\n", functions_cnt);
> +
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
>         return 0;
>  }
>
> @@ -312,7 +354,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
>         struct elf_function key = { .name = name };
>
>         return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> -                      functions_cmp);
> +                      functions_cmp_name);
>  }
>
>  static bool btf_name_char_ok(char c, bool first)
>

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Thu, Feb 11, 2021 at 11:59:02AM -0800, Andrii Nakryiko wrote:

SNIP

> >         return strcmp(a->name, b->name);
> >  }
> >
> > +static int functions_cmp_addr(const void *_a, const void *_b)
> > +{
> > +       const struct elf_function *a = _a;
> > +       const struct elf_function *b = _b;
> > +
> > +       if (a->addr == b->addr)
> > +               return 0;
> > +       return a->addr < b->addr ? -1 : 1;
> > +}
> > +
> >  static void delete_functions(void)
> >  {
> >         free(functions);
> > @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
> >
> >         functions[functions_cnt].name = name;
> >         functions[functions_cnt].addr = elf_sym__value(sym);
> > +       functions[functions_cnt].end = (__u64) -1;
> >         functions[functions_cnt].sh_addr = sh.sh_addr;
> >         functions[functions_cnt].generated = false;
> >         functions_cnt++;
> > @@ -236,6 +248,40 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
> >         return 0;
> >  }
> >
> > +static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
> 
> return bool, not int?

yep

> 
> > +                         __u64 count, bool kmod)
> > +{
> > +       /*
> > +        * For vmlinux image both addrs[x] and functions[x]::addr
> > +        * values are final address and are comparable.
> > +        *
> > +        * For kernel module addrs[x] is final address, but
> > +        * functions[x]::addr is relative address within section
> > +        * and needs to be relocated by adding sh_addr.
> > +        */
> > +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> > +       __u64 end   = kmod ? func->end + func->sh_addr : func->end;
> > +
> > +       size_t l = 0, r = count - 1, m;
> > +       __u64 addr = 0;
> > +
> > +       while (l < r) {
> > +               m = l + (r - l + 1) / 2;
> > +               addr = addrs[m];
> > +
> > +               if (start <= addr && addr < end)
> > +                       return true;
> 
> this extra check on each step shouldn't be necessary
> 
> > +
> > +               if (start <= addr)
> 
> I don't think this is correct, start == addr is actually a good case,
> but you'll do r = m - 1, skipping it. See below about invariants.

the == case is covered in the check above, but yes it should be <

> 
> > +                       r = m - 1;
> > +               else
> > +                       l = m;
> 
> So in my previous example I assumed we have address ranges for ftrace
> section, which is exactly the opposite from what we have. So this
> binary search should be a bit different. start <= addr seems wrong
> here as well.
> 
> The invariant here should be that addr[r] is the smallest address that
> is >= than function start addr, right? Except the corner case where
> there is no such r, but for that we have a final check in the return
> below. If you wanted to use index l, you'd need to change the
> invariant to find the largest addr, such that it is < end, but that
> seems a bit convoluted.
> 
> So, with that, I think it should be like this:
> 
> size_t l = 0, r = count - 1, m;
> 
> /* make sure we don't use invalid r */
> if (count == 0) return false;
> 
> while (l < r) {
>     /* note no +1 in this case, it's so that at the end, when you
>      * have, say, l = 0, and r = 1, you try l first, not r.
>      * Otherwise you might end in in the infinite loop when r never == l.
>      */

nice catch, did not see that

>     m = l + (r - l) / 2;
>     addr = addrs[m];
> 
>     if (addr >= start)
>         /* we satisfy invariant, so tighten r */
>         r = m;
>     else
>         /* m is not good enough as l, maybe m + 1 will be */
>         l = m + 1;
> }
> 
> return start <= addrs[r] && addrs[r] < end;
> 
> 
> So, basically, r is maintained as a valid index always, while we
> constantly try to tighten the l.
> 
> Does this make sense?

I did not see the possibility to let the search go through
all the way to l == r and that 'extra' check allowed me to split
the interval, without caring about invariant

but I think your solution is cleaner, I'll send new version

> 
> 
> > +       }
> > +
> > +       addr = addrs[l];
> > +       return start <= addr && addr < end;
> > +}
> > +
> >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >  {
> >         __u64 *addrs, count, i;
> > @@ -267,7 +313,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >         }
> >
> >         qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> > -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> > +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
> 
> See below assumptions about function end. If we get it from ELF, you
> don't need to do this extra sort, right?

I had the same assumption and was surprised why my code is not working ;-)

> 
> >
> >         /*
> >          * Let's got through all collected functions and filter
> > @@ -275,18 +321,12 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >          */
> >         for (i = 0; i < functions_cnt; i++) {
> >                 struct elf_function *func = &functions[i];
> > -               /*
> > -                * For vmlinux image both addrs[x] and functions[x]::addr
> > -                * values are final address and are comparable.
> > -                *
> > -                * For kernel module addrs[x] is final address, but
> > -                * functions[x]::addr is relative address within section
> > -                * and needs to be relocated by adding sh_addr.
> > -                */
> > -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
> > +
> > +               if (i + 1 < functions_cnt)
> > +                       func->end = functions[i + 1].addr;
> 
> This makes a bunch of unnecessary assumptions about functions layout.
> But why, if we have STT_FUNC symbol with function size, so that we
> know the function end right when we collect function info.

ugh forgot about the st_size for STT_FUNC

thanks,
jirka

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Thu, Feb 11, 2021 at 11:59:02AM -0800, Andrii Nakryiko wrote:

SNIP

> 
> So in my previous example I assumed we have address ranges for ftrace
> section, which is exactly the opposite from what we have. So this
> binary search should be a bit different. start <= addr seems wrong
> here as well.
> 
> The invariant here should be that addr[r] is the smallest address that
> is >= than function start addr, right? Except the corner case where
> there is no such r, but for that we have a final check in the return
> below. If you wanted to use index l, you'd need to change the
> invariant to find the largest addr, such that it is < end, but that
> seems a bit convoluted.
> 
> So, with that, I think it should be like this:
> 
> size_t l = 0, r = count - 1, m;
> 
> /* make sure we don't use invalid r */
> if (count == 0) return false;
> 
> while (l < r) {
>     /* note no +1 in this case, it's so that at the end, when you
>      * have, say, l = 0, and r = 1, you try l first, not r.
>      * Otherwise you might end in in the infinite loop when r never == l.
>      */
>     m = l + (r - l) / 2;
>     addr = addrs[m];
> 
>     if (addr >= start)
>         /* we satisfy invariant, so tighten r */
>         r = m;
>     else
>         /* m is not good enough as l, maybe m + 1 will be */
>         l = m + 1;
> }
> 
> return start <= addrs[r] && addrs[r] < end;
> 
> 
> So, basically, r is maintained as a valid index always, while we
> constantly try to tighten the l.
> 
> Does this make sense?

another take ;-)

jirka


---
diff --git a/btf_encoder.c b/btf_encoder.c
index b124ec20a689..20a93ed60e52 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -36,6 +36,7 @@ struct funcs_layout {
 struct elf_function {
 	const char	*name;
 	unsigned long	 addr;
+	unsigned long	 size;
 	unsigned long	 sh_addr;
 	bool		 generated;
 };
@@ -44,7 +45,7 @@ static struct elf_function *functions;
 static int functions_alloc;
 static int functions_cnt;
 
-static int functions_cmp(const void *_a, const void *_b)
+static int functions_cmp_name(const void *_a, const void *_b)
 {
 	const struct elf_function *a = _a;
 	const struct elf_function *b = _b;
@@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
 	return strcmp(a->name, b->name);
 }
 
+static int functions_cmp_addr(const void *_a, const void *_b)
+{
+	const struct elf_function *a = _a;
+	const struct elf_function *b = _b;
+
+	if (a->addr == b->addr)
+		return 0;
+	return a->addr < b->addr ? -1 : 1;
+}
+
 static void delete_functions(void)
 {
 	free(functions);
@@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
 
 	functions[functions_cnt].name = name;
 	functions[functions_cnt].addr = elf_sym__value(sym);
+	functions[functions_cnt].size = elf_sym__size(sym);
 	functions[functions_cnt].sh_addr = sh.sh_addr;
 	functions[functions_cnt].generated = false;
 	functions_cnt++;
@@ -236,6 +248,48 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
 	return 0;
 }
 
+static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
+			  __u64 count, bool kmod)
+{
+	/*
+	 * For vmlinux image both addrs[x] and functions[x]::addr
+	 * values are final address and are comparable.
+	 *
+	 * For kernel module addrs[x] is final address, but
+	 * functions[x]::addr is relative address within section
+	 * and needs to be relocated by adding sh_addr.
+	 */
+	__u64 start = kmod ? func->addr + func->sh_addr : func->addr;
+	__u64 addr, end = func->addr + func->size;
+
+	/*
+	 * The invariant here is addr[r] that is the smallest address
+	 * that is >= than function start addr. Except the corner case
+	 * where there is no such r, but for that we have a final check
+	 * in the return.
+	 */
+	size_t l = 0, r = count - 1, m;
+
+	/* make sure we don't use invalid r */
+	if (count == 0)
+		return false;
+
+	while (l < r) {
+		m = l + (r - l) / 2;
+		addr = addrs[m];
+
+		if (addr >= start) {
+			/* we satisfy invariant, so tighten r */
+			r = m;
+		} else {
+			/* m is not good enough as l, maybe m + 1 will be */
+			l = m + 1;
+		}
+	}
+
+	return start <= addrs[r] && addrs[r] < end;
+}
+
 static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 {
 	__u64 *addrs, count, i;
@@ -267,7 +321,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 	}
 
 	qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
-	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
+	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
 
 	/*
 	 * Let's got through all collected functions and filter
@@ -275,18 +329,9 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 	 */
 	for (i = 0; i < functions_cnt; i++) {
 		struct elf_function *func = &functions[i];
-		/*
-		 * For vmlinux image both addrs[x] and functions[x]::addr
-		 * values are final address and are comparable.
-		 *
-		 * For kernel module addrs[x] is final address, but
-		 * functions[x]::addr is relative address within section
-		 * and needs to be relocated by adding sh_addr.
-		 */
-		__u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
 
 		/* Make sure function is within ftrace addresses. */
-		if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
+		if (is_ftrace_func(func, addrs, count, kmod)) {
 			/*
 			 * We iterate over sorted array, so we can easily skip
 			 * not valid item and move following valid field into
@@ -303,6 +348,8 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
 
 	if (btf_elf__verbose)
 		printf("Found %d functions!\n", functions_cnt);
+
+	qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
 	return 0;
 }
 
@@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
 	struct elf_function key = { .name = name };
 
 	return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
-		       functions_cmp);
+		       functions_cmp_name);
 }
 
 static bool btf_name_char_ok(char c, bool first)

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Andrii Nakryiko]

On Fri, Feb 12, 2021 at 8:39 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Thu, Feb 11, 2021 at 11:59:02AM -0800, Andrii Nakryiko wrote:
>
> SNIP
>
> >
> > So in my previous example I assumed we have address ranges for ftrace
> > section, which is exactly the opposite from what we have. So this
> > binary search should be a bit different. start <= addr seems wrong
> > here as well.
> >
> > The invariant here should be that addr[r] is the smallest address that
> > is >= than function start addr, right? Except the corner case where
> > there is no such r, but for that we have a final check in the return
> > below. If you wanted to use index l, you'd need to change the
> > invariant to find the largest addr, such that it is < end, but that
> > seems a bit convoluted.
> >
> > So, with that, I think it should be like this:
> >
> > size_t l = 0, r = count - 1, m;
> >
> > /* make sure we don't use invalid r */
> > if (count == 0) return false;
> >
> > while (l < r) {
> >     /* note no +1 in this case, it's so that at the end, when you
> >      * have, say, l = 0, and r = 1, you try l first, not r.
> >      * Otherwise you might end in in the infinite loop when r never == l.
> >      */
> >     m = l + (r - l) / 2;
> >     addr = addrs[m];
> >
> >     if (addr >= start)
> >         /* we satisfy invariant, so tighten r */
> >         r = m;
> >     else
> >         /* m is not good enough as l, maybe m + 1 will be */
> >         l = m + 1;
> > }
> >
> > return start <= addrs[r] && addrs[r] < end;
> >
> >
> > So, basically, r is maintained as a valid index always, while we
> > constantly try to tighten the l.
> >
> > Does this make sense?
>
> another take ;-)
>
> jirka
>
>
> ---
> diff --git a/btf_encoder.c b/btf_encoder.c
> index b124ec20a689..20a93ed60e52 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -36,6 +36,7 @@ struct funcs_layout {
>  struct elf_function {
>         const char      *name;
>         unsigned long    addr;
> +       unsigned long    size;
>         unsigned long    sh_addr;
>         bool             generated;
>  };
> @@ -44,7 +45,7 @@ static struct elf_function *functions;
>  static int functions_alloc;
>  static int functions_cnt;
>
> -static int functions_cmp(const void *_a, const void *_b)
> +static int functions_cmp_name(const void *_a, const void *_b)
>  {
>         const struct elf_function *a = _a;
>         const struct elf_function *b = _b;
> @@ -52,6 +53,16 @@ static int functions_cmp(const void *_a, const void *_b)
>         return strcmp(a->name, b->name);
>  }
>
> +static int functions_cmp_addr(const void *_a, const void *_b)
> +{
> +       const struct elf_function *a = _a;
> +       const struct elf_function *b = _b;
> +
> +       if (a->addr == b->addr)
> +               return 0;
> +       return a->addr < b->addr ? -1 : 1;
> +}
> +
>  static void delete_functions(void)
>  {
>         free(functions);
> @@ -98,6 +109,7 @@ static int collect_function(struct btf_elf *btfe, GElf_Sym *sym,
>
>         functions[functions_cnt].name = name;
>         functions[functions_cnt].addr = elf_sym__value(sym);
> +       functions[functions_cnt].size = elf_sym__size(sym);
>         functions[functions_cnt].sh_addr = sh.sh_addr;
>         functions[functions_cnt].generated = false;
>         functions_cnt++;
> @@ -236,6 +248,48 @@ get_kmod_addrs(struct btf_elf *btfe, __u64 **paddrs, __u64 *pcount)
>         return 0;
>  }
>
> +static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
> +                         __u64 count, bool kmod)
> +{
> +       /*
> +        * For vmlinux image both addrs[x] and functions[x]::addr
> +        * values are final address and are comparable.
> +        *
> +        * For kernel module addrs[x] is final address, but
> +        * functions[x]::addr is relative address within section
> +        * and needs to be relocated by adding sh_addr.
> +        */
> +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> +       __u64 addr, end = func->addr + func->size;
> +
> +       /*
> +        * The invariant here is addr[r] that is the smallest address
> +        * that is >= than function start addr. Except the corner case
> +        * where there is no such r, but for that we have a final check
> +        * in the return.
> +        */
> +       size_t l = 0, r = count - 1, m;
> +
> +       /* make sure we don't use invalid r */
> +       if (count == 0)
> +               return false;
> +
> +       while (l < r) {
> +               m = l + (r - l) / 2;
> +               addr = addrs[m];
> +
> +               if (addr >= start) {
> +                       /* we satisfy invariant, so tighten r */
> +                       r = m;
> +               } else {
> +                       /* m is not good enough as l, maybe m + 1 will be */
> +                       l = m + 1;
> +               }
> +       }
> +
> +       return start <= addrs[r] && addrs[r] < end;
> +}
> +
>  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>  {
>         __u64 *addrs, count, i;
> @@ -267,7 +321,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>         }
>
>         qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);

All looks good except this. We don't rely on functions being sorted in
ascending start addr order, do we? If not, just drop this, no need to
slow down the process.

>
>         /*
>          * Let's got through all collected functions and filter
> @@ -275,18 +329,9 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>          */
>         for (i = 0; i < functions_cnt; i++) {
>                 struct elf_function *func = &functions[i];
> -               /*
> -                * For vmlinux image both addrs[x] and functions[x]::addr
> -                * values are final address and are comparable.
> -                *
> -                * For kernel module addrs[x] is final address, but
> -                * functions[x]::addr is relative address within section
> -                * and needs to be relocated by adding sh_addr.
> -                */
> -               __u64 addr = kmod ? func->addr + func->sh_addr : func->addr;
>
>                 /* Make sure function is within ftrace addresses. */
> -               if (bsearch(&addr, addrs, count, sizeof(addrs[0]), addrs_cmp)) {
> +               if (is_ftrace_func(func, addrs, count, kmod)) {
>                         /*
>                          * We iterate over sorted array, so we can easily skip
>                          * not valid item and move following valid field into
> @@ -303,6 +348,8 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
>
>         if (btf_elf__verbose)
>                 printf("Found %d functions!\n", functions_cnt);
> +
> +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_name);
>         return 0;
>  }
>
> @@ -312,7 +359,7 @@ static struct elf_function *find_function(const struct btf_elf *btfe,
>         struct elf_function key = { .name = name };
>
>         return bsearch(&key, functions, functions_cnt, sizeof(functions[0]),
> -                      functions_cmp);
> +                      functions_cmp_name);
>  }
>
>  static bool btf_name_char_ok(char c, bool first)
>

Re: FAILED unresolved symbol vfs_truncate on arm64 with LLVM

[Jiri Olsa]

On Fri, Feb 12, 2021 at 11:22:41AM -0800, Andrii Nakryiko wrote:

SNIP

> > +static int is_ftrace_func(struct elf_function *func, __u64 *addrs,
> > +                         __u64 count, bool kmod)
> > +{
> > +       /*
> > +        * For vmlinux image both addrs[x] and functions[x]::addr
> > +        * values are final address and are comparable.
> > +        *
> > +        * For kernel module addrs[x] is final address, but
> > +        * functions[x]::addr is relative address within section
> > +        * and needs to be relocated by adding sh_addr.
> > +        */
> > +       __u64 start = kmod ? func->addr + func->sh_addr : func->addr;
> > +       __u64 addr, end = func->addr + func->size;
> > +
> > +       /*
> > +        * The invariant here is addr[r] that is the smallest address
> > +        * that is >= than function start addr. Except the corner case
> > +        * where there is no such r, but for that we have a final check
> > +        * in the return.
> > +        */
> > +       size_t l = 0, r = count - 1, m;
> > +
> > +       /* make sure we don't use invalid r */
> > +       if (count == 0)
> > +               return false;
> > +
> > +       while (l < r) {
> > +               m = l + (r - l) / 2;
> > +               addr = addrs[m];
> > +
> > +               if (addr >= start) {
> > +                       /* we satisfy invariant, so tighten r */
> > +                       r = m;
> > +               } else {
> > +                       /* m is not good enough as l, maybe m + 1 will be */
> > +                       l = m + 1;
> > +               }
> > +       }
> > +
> > +       return start <= addrs[r] && addrs[r] < end;
> > +}
> > +
> >  static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >  {
> >         __u64 *addrs, count, i;
> > @@ -267,7 +321,7 @@ static int setup_functions(struct btf_elf *btfe, struct funcs_layout *fl)
> >         }
> >
> >         qsort(addrs, count, sizeof(addrs[0]), addrs_cmp);
> > -       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp);
> > +       qsort(functions, functions_cnt, sizeof(functions[0]), functions_cmp_addr);
> 
> All looks good except this. We don't rely on functions being sorted in
> ascending start addr order, do we? If not, just drop this, no need to
> slow down the process.

right, it's not needed when we use st_size for function size

thanks,
jirka