* Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 @ 2020-10-18 23:18 Érico Rolim 2020-10-20 9:01 ` Jiri Slaby 0 siblings, 1 reply; 14+ messages in thread From: Érico Rolim @ 2020-10-18 23:18 UTC (permalink / raw) To: dwarves, linux-kernel Hi! I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole version 1.18 segfaults during the build, as can be seen below: PAHOLE: Error: Found symbol of zero size when encoding btf (sym: '__kvm_nvhe_arm64_ssbd_callback_required', cu: 'arch/arm64/kernel/cpu_errata.c'). PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force emit the btf. scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} LD .tmp_vmlinux.kallsyms1 KSYM .tmp_vmlinux.kallsyms1.o LD .tmp_vmlinux.kallsyms2 KSYM .tmp_vmlinux.kallsyms2.o LD vmlinux BTFIDS vmlinux FAILED: load BTF from vmlinux: Unknown error -2make: *** [Makefile:1162: vmlinux] Error 255 It is possible to force the build to continue if LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} in scripts/link-vmlinux.sh is changed to LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} The suggested `-j` or `--force` flags don't exist, since they were removed in [1]. I believe `--btf_encode_force` should be suggested instead. It should be noted that the same build, but with pahole version 1.17, works without issue, so I think this is either a regression in pahole or the script will need to be changed for newer versions of pahole. - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 Thanks, Érico ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-18 23:18 Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 Érico Rolim @ 2020-10-20 9:01 ` Jiri Slaby 2020-10-20 12:20 ` Arnaldo Carvalho de Melo 2020-10-20 17:15 ` Andrii Nakryiko 0 siblings, 2 replies; 14+ messages in thread From: Jiri Slaby @ 2020-10-20 9:01 UTC (permalink / raw) To: Érico Rolim, dwarves, linux-kernel, Arnaldo Carvalho de Melo, Hao Luo Hi, On 19. 10. 20, 1:18, Érico Rolim wrote: > I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has > `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole > version 1.18 segfaults during the build, as can be seen below: > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: > '__kvm_nvhe_arm64_ssbd_callback_required', cu: > 'arch/arm64/kernel/cpu_errata.c'). The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; > PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force > emit the btf. > scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > LD .tmp_vmlinux.kallsyms1 > KSYM .tmp_vmlinux.kallsyms1.o > LD .tmp_vmlinux.kallsyms2 > KSYM .tmp_vmlinux.kallsyms2.o > LD vmlinux > BTFIDS vmlinux > FAILED: load BTF from vmlinux: Unknown error -2make: *** > [Makefile:1162: vmlinux] Error 255 > > It is possible to force the build to continue if > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > in scripts/link-vmlinux.sh is changed to > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} > > The suggested `-j` or `--force` flags don't exist, since they were removed in > [1]. I believe `--btf_encode_force` should be suggested instead. Agreed, '--btf_encode_force' makes pahole to proceed without crashes. > It should be noted that the same build, but with pahole version 1.17, works > without issue, so I think this is either a regression in pahole or the script > will need to be changed for newer versions of pahole. Yeah, I observe the very same. I reported it at: https://bugzilla.suse.com/show_bug.cgi?id=1177921 The backtrace: > (gdb) where > #0 __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:300 > #1 0x00007ffff7f78346 in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>, __dest=<optimized out>, __src=<optimized out>, __len=<optimized out>) at /usr/include/bits/string_fortified.h:34 > #2 gobuffer__add (gb=0x555555569aa0, s=0x7fffffffb50c, len=12) at /usr/src/debug/dwarves-1.18-1.1.x86_64/gobuffer.c:87 > #3 0x00007ffff7f8671f in btf_elf__add_datasec_type (btfe=btfe@entry=0x555555569a40, section_name=section_name@entry=0x7ffff7fa43ad ".data..percpu", var_secinfo_buf=var_secinfo_buf@entry=0x555555569ac0) at /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.c:721 > #4 0x00007ffff7f8d766 in btf_elf__encode (flags=0 '\000', btfe=0x555555569a40) at /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.c:857 > #5 btf_elf__encode (btfe=0x555555569a40, flags=<optimized out>) at /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.h:71 > #6 0x00007ffff7f7fc70 in btf_encoder__encode () at /usr/src/debug/dwarves-1.18-1.1.x86_64/btf_encoder.c:213 > #7 0x00007ffff7f80d17 in cu__encode_btf (cu=0x55555638d9b0, verbose=0, force=false, skip_encoding_vars=false) at /usr/src/debug/dwarves-1.18-1.1.x86_64/btf_encoder.c:255 > #8 0x000055555555ac4d in pahole_stealer (cu=0x55555638d9b0, conf_load=<optimized out>) at /usr/src/debug/dwarves-1.18-1.1.x86_64/pahole.c:2366 > #9 0x00007ffff7f89dab in finalize_cu (cus=0x5555555622d0, dcu=0x7fffffffd080, conf=0x5555555610e0 <conf_load>, cu=0x55555638d9b0) at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2473 > #10 finalize_cu_immediately (conf=0x5555555610e0 <conf_load>, dcu=0x7fffffffd080, cu=0x55555638d9b0, cus=0x5555555622d0) at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2317 > #11 cus__load_module (cus=cus@entry=0x5555555622d0, conf=0x5555555610e0 <conf_load>, mod=mod@entry=0x555555564760, dw=0x555555565960, elf=elf@entry=0x555555562360, filename=0x7fffffffe846 "ss") at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2473 > #12 0x00007ffff7f8a0f1 in cus__process_dwflmod (dwflmod=0x555555564760, userdata=<optimized out>, name=<optimized out>, base=<optimized out>, arg=0x7fffffffe1b0) at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2518 > #13 0x00007ffff7d4f571 in dwfl_getmodules () from /usr/lib64/libdw.so.1 > #14 0x00007ffff7f823ed in cus__process_file (filename=0x7fffffffe846 "ss", fd=3, conf=<optimized out>, cus=0x5555555622d0) at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2571 > #15 dwarf__load_file (cus=0x5555555622d0, conf=<optimized out>, filename=0x7fffffffe846 "ss") at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2588 > #16 0x00007ffff7f76771 in cus__load_file (cus=cus@entry=0x5555555622d0, conf=conf@entry=0x5555555610e0 <conf_load>, filename=0x7fffffffe846 "ss") at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarves.c:1958 > #17 0x00007ffff7f798a8 in cus__load_files (cus=0x5555555622d0, conf=0x5555555610e0 <conf_load>, filenames=0x7fffffffe518) at /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarves.c:2316 > #18 0x00005555555576fc in main (argc=3, argv=0x7fffffffe508) at /usr/src/debug/dwarves-1.18-1.1.x86_64/pahole.c:2687 I suspect: commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 Author: Hao Luo <haoluo@google.com> AuthorDate: Wed Jul 8 13:44:10 2020 -0700 btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. Which added this machinery (btf_elf__add_datasec_type in particular). > - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 > > Thanks, > Érico > -- js suse labs ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 9:01 ` Jiri Slaby @ 2020-10-20 12:20 ` Arnaldo Carvalho de Melo 2020-10-20 17:04 ` Hao Luo 2020-10-21 6:22 ` Jiri Slaby 2020-10-20 17:15 ` Andrii Nakryiko 1 sibling, 2 replies; 14+ messages in thread From: Arnaldo Carvalho de Melo @ 2020-10-20 12:20 UTC (permalink / raw) To: Jiri Slaby Cc: Érico Rolim, dwarves, linux-kernel, Arnaldo Carvalho de Melo, Hao Luo, Andrii Nakryiko Em Tue, Oct 20, 2020 at 11:01:39AM +0200, Jiri Slaby escreveu: > Hi, > > On 19. 10. 20, 1:18, Érico Rolim wrote: > > I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has > > `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole > > version 1.18 segfaults during the build, as can be seen below: > > > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: > > '__kvm_nvhe_arm64_ssbd_callback_required', cu: > > 'arch/arm64/kernel/cpu_errata.c'). > > The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: > __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; > > > PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force > > emit the btf. > > scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > LD .tmp_vmlinux.kallsyms1 > > KSYM .tmp_vmlinux.kallsyms1.o > > LD .tmp_vmlinux.kallsyms2 > > KSYM .tmp_vmlinux.kallsyms2.o > > LD vmlinux > > BTFIDS vmlinux > > FAILED: load BTF from vmlinux: Unknown error -2make: *** > > [Makefile:1162: vmlinux] Error 255 > > > > It is possible to force the build to continue if > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > in scripts/link-vmlinux.sh is changed to > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} > > > > The suggested `-j` or `--force` flags don't exist, since they were removed in > > [1]. I believe `--btf_encode_force` should be suggested instead. > > Agreed, '--btf_encode_force' makes pahole to proceed without crashes. > > > It should be noted that the same build, but with pahole version 1.17, works > > without issue, so I think this is either a regression in pahole or the script > > will need to be changed for newer versions of pahole. > > Yeah, I observe the very same. I reported it at: > https://bugzilla.suse.com/show_bug.cgi?id=1177921 Would it be possible to try with https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder ? This switches to using libbpf for the BTF encoder and may have fixed this problem. - Arnaldo > The backtrace: > > (gdb) where > > #0 __memmove_sse2_unaligned_erms () at > ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:300 > > #1 0x00007ffff7f78346 in memcpy (__len=<optimized out>, __src=<optimized > out>, __dest=<optimized out>, __dest=<optimized out>, __src=<optimized out>, > __len=<optimized out>) at /usr/include/bits/string_fortified.h:34 > > #2 gobuffer__add (gb=0x555555569aa0, s=0x7fffffffb50c, len=12) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/gobuffer.c:87 > > #3 0x00007ffff7f8671f in btf_elf__add_datasec_type > (btfe=btfe@entry=0x555555569a40, > section_name=section_name@entry=0x7ffff7fa43ad ".data..percpu", > var_secinfo_buf=var_secinfo_buf@entry=0x555555569ac0) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.c:721 > > #4 0x00007ffff7f8d766 in btf_elf__encode (flags=0 '\000', > btfe=0x555555569a40) at /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.c:857 > > #5 btf_elf__encode (btfe=0x555555569a40, flags=<optimized out>) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.h:71 > > #6 0x00007ffff7f7fc70 in btf_encoder__encode () at > /usr/src/debug/dwarves-1.18-1.1.x86_64/btf_encoder.c:213 > > #7 0x00007ffff7f80d17 in cu__encode_btf (cu=0x55555638d9b0, verbose=0, > force=false, skip_encoding_vars=false) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/btf_encoder.c:255 > > #8 0x000055555555ac4d in pahole_stealer (cu=0x55555638d9b0, > conf_load=<optimized out>) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/pahole.c:2366 > > #9 0x00007ffff7f89dab in finalize_cu (cus=0x5555555622d0, > dcu=0x7fffffffd080, conf=0x5555555610e0 <conf_load>, cu=0x55555638d9b0) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2473 > > #10 finalize_cu_immediately (conf=0x5555555610e0 <conf_load>, > dcu=0x7fffffffd080, cu=0x55555638d9b0, cus=0x5555555622d0) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2317 > > #11 cus__load_module (cus=cus@entry=0x5555555622d0, conf=0x5555555610e0 > <conf_load>, mod=mod@entry=0x555555564760, dw=0x555555565960, > elf=elf@entry=0x555555562360, filename=0x7fffffffe846 "ss") at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2473 > > #12 0x00007ffff7f8a0f1 in cus__process_dwflmod (dwflmod=0x555555564760, > userdata=<optimized out>, name=<optimized out>, base=<optimized out>, > arg=0x7fffffffe1b0) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2518 > > #13 0x00007ffff7d4f571 in dwfl_getmodules () from /usr/lib64/libdw.so.1 > > #14 0x00007ffff7f823ed in cus__process_file (filename=0x7fffffffe846 "ss", > fd=3, conf=<optimized out>, cus=0x5555555622d0) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2571 > > #15 dwarf__load_file (cus=0x5555555622d0, conf=<optimized out>, > filename=0x7fffffffe846 "ss") at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2588 > > #16 0x00007ffff7f76771 in cus__load_file (cus=cus@entry=0x5555555622d0, > conf=conf@entry=0x5555555610e0 <conf_load>, filename=0x7fffffffe846 "ss") at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarves.c:1958 > > #17 0x00007ffff7f798a8 in cus__load_files (cus=0x5555555622d0, > conf=0x5555555610e0 <conf_load>, filenames=0x7fffffffe518) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarves.c:2316 > > #18 0x00005555555576fc in main (argc=3, argv=0x7fffffffe508) at > /usr/src/debug/dwarves-1.18-1.1.x86_64/pahole.c:2687 > > > I suspect: > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > Author: Hao Luo <haoluo@google.com> > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > Which added this machinery (btf_elf__add_datasec_type in particular). > > > - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 > > > > Thanks, > > Érico > > > > > -- > js > suse labs -- - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 12:20 ` Arnaldo Carvalho de Melo @ 2020-10-20 17:04 ` Hao Luo 2020-10-20 17:10 ` Andrii Nakryiko 2020-10-21 6:22 ` Jiri Slaby 1 sibling, 1 reply; 14+ messages in thread From: Hao Luo @ 2020-10-20 17:04 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Andrii Nakryiko, Alexei Starovoitov, bpf, Andrii Nakryiko Thanks for reporting this and cc'ing me. I forgot to update the error messages when renaming the flags. I will send a patch to fix the error message. The commit commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 Author: Hao Luo <haoluo@google.com> AuthorDate: Wed Jul 8 13:44:10 2020 -0700 btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. encodes kernel global variables into BTF so that bpf programs can directly access them. If there is no need to access kernel global variables, it's perfectly fine to use '--btf_encode_force' to skip encoding bad symbols into BTF, or '--skip_encoding_btf_vars' to skip encoding all global vars all together. I will add these info into the updated error message. Also cc bpf folks for attention of this bug. Hao On Tue, Oct 20, 2020 at 5:20 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote: > > Em Tue, Oct 20, 2020 at 11:01:39AM +0200, Jiri Slaby escreveu: > > Hi, > > > > On 19. 10. 20, 1:18, Érico Rolim wrote: > > > I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has > > > `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole > > > version 1.18 segfaults during the build, as can be seen below: > > > > > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: > > > '__kvm_nvhe_arm64_ssbd_callback_required', cu: > > > 'arch/arm64/kernel/cpu_errata.c'). > > > > The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: > > __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; > > > > > PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force > > > emit the btf. > > > scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > LD .tmp_vmlinux.kallsyms1 > > > KSYM .tmp_vmlinux.kallsyms1.o > > > LD .tmp_vmlinux.kallsyms2 > > > KSYM .tmp_vmlinux.kallsyms2.o > > > LD vmlinux > > > BTFIDS vmlinux > > > FAILED: load BTF from vmlinux: Unknown error -2make: *** > > > [Makefile:1162: vmlinux] Error 255 > > > > > > It is possible to force the build to continue if > > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > > > in scripts/link-vmlinux.sh is changed to > > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} > > > > > > The suggested `-j` or `--force` flags don't exist, since they were removed in > > > [1]. I believe `--btf_encode_force` should be suggested instead. > > > > Agreed, '--btf_encode_force' makes pahole to proceed without crashes. > > > > > It should be noted that the same build, but with pahole version 1.17, works > > > without issue, so I think this is either a regression in pahole or the script > > > will need to be changed for newer versions of pahole. > > > > Yeah, I observe the very same. I reported it at: > > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > Would it be possible to try with > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > ? > > This switches to using libbpf for the BTF encoder and may have fixed > this problem. > > - Arnaldo > > > The backtrace: > > > (gdb) where > > > #0 __memmove_sse2_unaligned_erms () at > > ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:300 > > > #1 0x00007ffff7f78346 in memcpy (__len=<optimized out>, __src=<optimized > > out>, __dest=<optimized out>, __dest=<optimized out>, __src=<optimized out>, > > __len=<optimized out>) at /usr/include/bits/string_fortified.h:34 > > > #2 gobuffer__add (gb=0x555555569aa0, s=0x7fffffffb50c, len=12) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/gobuffer.c:87 > > > #3 0x00007ffff7f8671f in btf_elf__add_datasec_type > > (btfe=btfe@entry=0x555555569a40, > > section_name=section_name@entry=0x7ffff7fa43ad ".data..percpu", > > var_secinfo_buf=var_secinfo_buf@entry=0x555555569ac0) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.c:721 > > > #4 0x00007ffff7f8d766 in btf_elf__encode (flags=0 '\000', > > btfe=0x555555569a40) at /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.c:857 > > > #5 btf_elf__encode (btfe=0x555555569a40, flags=<optimized out>) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/libbtf.h:71 > > > #6 0x00007ffff7f7fc70 in btf_encoder__encode () at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/btf_encoder.c:213 > > > #7 0x00007ffff7f80d17 in cu__encode_btf (cu=0x55555638d9b0, verbose=0, > > force=false, skip_encoding_vars=false) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/btf_encoder.c:255 > > > #8 0x000055555555ac4d in pahole_stealer (cu=0x55555638d9b0, > > conf_load=<optimized out>) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/pahole.c:2366 > > > #9 0x00007ffff7f89dab in finalize_cu (cus=0x5555555622d0, > > dcu=0x7fffffffd080, conf=0x5555555610e0 <conf_load>, cu=0x55555638d9b0) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2473 > > > #10 finalize_cu_immediately (conf=0x5555555610e0 <conf_load>, > > dcu=0x7fffffffd080, cu=0x55555638d9b0, cus=0x5555555622d0) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2317 > > > #11 cus__load_module (cus=cus@entry=0x5555555622d0, conf=0x5555555610e0 > > <conf_load>, mod=mod@entry=0x555555564760, dw=0x555555565960, > > elf=elf@entry=0x555555562360, filename=0x7fffffffe846 "ss") at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2473 > > > #12 0x00007ffff7f8a0f1 in cus__process_dwflmod (dwflmod=0x555555564760, > > userdata=<optimized out>, name=<optimized out>, base=<optimized out>, > > arg=0x7fffffffe1b0) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2518 > > > #13 0x00007ffff7d4f571 in dwfl_getmodules () from /usr/lib64/libdw.so.1 > > > #14 0x00007ffff7f823ed in cus__process_file (filename=0x7fffffffe846 "ss", > > fd=3, conf=<optimized out>, cus=0x5555555622d0) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2571 > > > #15 dwarf__load_file (cus=0x5555555622d0, conf=<optimized out>, > > filename=0x7fffffffe846 "ss") at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarf_loader.c:2588 > > > #16 0x00007ffff7f76771 in cus__load_file (cus=cus@entry=0x5555555622d0, > > conf=conf@entry=0x5555555610e0 <conf_load>, filename=0x7fffffffe846 "ss") at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarves.c:1958 > > > #17 0x00007ffff7f798a8 in cus__load_files (cus=0x5555555622d0, > > conf=0x5555555610e0 <conf_load>, filenames=0x7fffffffe518) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/dwarves.c:2316 > > > #18 0x00005555555576fc in main (argc=3, argv=0x7fffffffe508) at > > /usr/src/debug/dwarves-1.18-1.1.x86_64/pahole.c:2687 > > > > > > I suspect: > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > > Author: Hao Luo <haoluo@google.com> > > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > > > > Which added this machinery (btf_elf__add_datasec_type in particular). > > > > > - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 > > > > > > Thanks, > > > Érico > > > > > > > > > -- > > js > > suse labs > > -- > > - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 17:04 ` Hao Luo @ 2020-10-20 17:10 ` Andrii Nakryiko 2020-10-20 17:18 ` Hao Luo 2020-10-20 18:14 ` Arnaldo Carvalho de Melo 0 siblings, 2 replies; 14+ messages in thread From: Andrii Nakryiko @ 2020-10-20 17:10 UTC (permalink / raw) To: Hao Luo Cc: Arnaldo Carvalho de Melo, Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Andrii Nakryiko, Alexei Starovoitov, bpf On Tue, Oct 20, 2020 at 10:05 AM Hao Luo <haoluo@google.com> wrote: > > Thanks for reporting this and cc'ing me. I forgot to update the error > messages when renaming the flags. I will send a patch to fix the error > message. > > The commit > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > Author: Hao Luo <haoluo@google.com> > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > encodes kernel global variables into BTF so that bpf programs can > directly access them. If there is no need to access kernel global > variables, it's perfectly fine to use '--btf_encode_force' to skip > encoding bad symbols into BTF, or '--skip_encoding_btf_vars' to skip > encoding all global vars all together. I will add these info into the > updated error message. > > Also cc bpf folks for attention of this bug. I've already fixed the message as part of 2e719cca6672 ("btf_encoder: revamp how per-CPU variables are encoded") It's currently still in the tmp.libbtf_encoder branch in pahole repo. > > Hao > > On Tue, Oct 20, 2020 at 5:20 AM Arnaldo Carvalho de Melo > <acme@kernel.org> wrote: > > > > Em Tue, Oct 20, 2020 at 11:01:39AM +0200, Jiri Slaby escreveu: > > > Hi, > > > > > > On 19. 10. 20, 1:18, Érico Rolim wrote: > > > > I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has > > > > `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole > > > > version 1.18 segfaults during the build, as can be seen below: > > > > > > > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: > > > > '__kvm_nvhe_arm64_ssbd_callback_required', cu: > > > > 'arch/arm64/kernel/cpu_errata.c'). > > > > > > The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: > > > __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; > > > > > > > PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force > > > > emit the btf. > > > > scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > LD .tmp_vmlinux.kallsyms1 > > > > KSYM .tmp_vmlinux.kallsyms1.o > > > > LD .tmp_vmlinux.kallsyms2 > > > > KSYM .tmp_vmlinux.kallsyms2.o > > > > LD vmlinux > > > > BTFIDS vmlinux > > > > FAILED: load BTF from vmlinux: Unknown error -2make: *** > > > > [Makefile:1162: vmlinux] Error 255 > > > > > > > > It is possible to force the build to continue if > > > > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > > > > > in scripts/link-vmlinux.sh is changed to > > > > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} > > > > > > > > The suggested `-j` or `--force` flags don't exist, since they were removed in > > > > [1]. I believe `--btf_encode_force` should be suggested instead. > > > > > > Agreed, '--btf_encode_force' makes pahole to proceed without crashes. > > > > > > > It should be noted that the same build, but with pahole version 1.17, works > > > > without issue, so I think this is either a regression in pahole or the script > > > > will need to be changed for newer versions of pahole. > > > > > > Yeah, I observe the very same. I reported it at: > > > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > > > Would it be possible to try with > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > > ? > > > > This switches to using libbpf for the BTF encoder and may have fixed > > this problem. > > > > - Arnaldo > > [...] > > > > > > > > > I suspect: > > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > > > Author: Hao Luo <haoluo@google.com> > > > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > > > > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > > > > > > > Which added this machinery (btf_elf__add_datasec_type in particular). > > > > > > > - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 > > > > > > > > Thanks, > > > > Érico > > > > > > > > > > > > > -- > > > js > > > suse labs > > > > -- > > > > - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 17:10 ` Andrii Nakryiko @ 2020-10-20 17:18 ` Hao Luo 2020-10-20 18:14 ` Arnaldo Carvalho de Melo 1 sibling, 0 replies; 14+ messages in thread From: Hao Luo @ 2020-10-20 17:18 UTC (permalink / raw) To: Andrii Nakryiko Cc: Arnaldo Carvalho de Melo, Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Andrii Nakryiko, Alexei Starovoitov, bpf On Tue, Oct 20, 2020 at 10:10 AM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote: > > On Tue, Oct 20, 2020 at 10:05 AM Hao Luo <haoluo@google.com> wrote: > > > > Thanks for reporting this and cc'ing me. I forgot to update the error > > messages when renaming the flags. I will send a patch to fix the error > > message. > > > > The commit > > > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > > Author: Hao Luo <haoluo@google.com> > > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > > encodes kernel global variables into BTF so that bpf programs can > > directly access them. If there is no need to access kernel global > > variables, it's perfectly fine to use '--btf_encode_force' to skip > > encoding bad symbols into BTF, or '--skip_encoding_btf_vars' to skip > > encoding all global vars all together. I will add these info into the > > updated error message. > > > > Also cc bpf folks for attention of this bug. > > I've already fixed the message as part of > 2e719cca6672 ("btf_encoder: revamp how per-CPU variables are encoded") > Ah, that's awesome! Thanks for fixing this, Andrii. I haven't got time to take a look at your patches last week, I will try to look at it ASAP. > It's currently still in the tmp.libbtf_encoder branch in pahole repo. > > > > > Hao > > > > On Tue, Oct 20, 2020 at 5:20 AM Arnaldo Carvalho de Melo > > <acme@kernel.org> wrote: > > > > > > Em Tue, Oct 20, 2020 at 11:01:39AM +0200, Jiri Slaby escreveu: > > > > Hi, > > > > > > > > On 19. 10. 20, 1:18, Érico Rolim wrote: > > > > > I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has > > > > > `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole > > > > > version 1.18 segfaults during the build, as can be seen below: > > > > > > > > > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: > > > > > '__kvm_nvhe_arm64_ssbd_callback_required', cu: > > > > > 'arch/arm64/kernel/cpu_errata.c'). > > > > > > > > The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: > > > > __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; > > > > > > > > > PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force > > > > > emit the btf. > > > > > scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > > LD .tmp_vmlinux.kallsyms1 > > > > > KSYM .tmp_vmlinux.kallsyms1.o > > > > > LD .tmp_vmlinux.kallsyms2 > > > > > KSYM .tmp_vmlinux.kallsyms2.o > > > > > LD vmlinux > > > > > BTFIDS vmlinux > > > > > FAILED: load BTF from vmlinux: Unknown error -2make: *** > > > > > [Makefile:1162: vmlinux] Error 255 > > > > > > > > > > It is possible to force the build to continue if > > > > > > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > > > > > > > in scripts/link-vmlinux.sh is changed to > > > > > > > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} > > > > > > > > > > The suggested `-j` or `--force` flags don't exist, since they were removed in > > > > > [1]. I believe `--btf_encode_force` should be suggested instead. > > > > > > > > Agreed, '--btf_encode_force' makes pahole to proceed without crashes. > > > > > > > > > It should be noted that the same build, but with pahole version 1.17, works > > > > > without issue, so I think this is either a regression in pahole or the script > > > > > will need to be changed for newer versions of pahole. > > > > > > > > Yeah, I observe the very same. I reported it at: > > > > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > > > > > Would it be possible to try with > > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > > > ? > > > > > > This switches to using libbpf for the BTF encoder and may have fixed > > > this problem. > > > > > > - Arnaldo > > > > > [...] > > > > > > > > > > > > > I suspect: > > > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > > > > Author: Hao Luo <haoluo@google.com> > > > > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > > > > > > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > > > > > > > > > > Which added this machinery (btf_elf__add_datasec_type in particular). > > > > > > > > > - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 > > > > > > > > > > Thanks, > > > > > Érico > > > > > > > > > > > > > > > > > -- > > > > js > > > > suse labs > > > > > > -- > > > > > > - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 17:10 ` Andrii Nakryiko 2020-10-20 17:18 ` Hao Luo @ 2020-10-20 18:14 ` Arnaldo Carvalho de Melo 2020-10-20 19:02 ` Arnaldo Carvalho de Melo 1 sibling, 1 reply; 14+ messages in thread From: Arnaldo Carvalho de Melo @ 2020-10-20 18:14 UTC (permalink / raw) To: Andrii Nakryiko Cc: Hao Luo, Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Andrii Nakryiko, Alexei Starovoitov, bpf Em Tue, Oct 20, 2020 at 10:10:19AM -0700, Andrii Nakryiko escreveu: > On Tue, Oct 20, 2020 at 10:05 AM Hao Luo <haoluo@google.com> wrote: > > Thanks for reporting this and cc'ing me. I forgot to update the error > > messages when renaming the flags. I will send a patch to fix the error > > message. > > The commit > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > > Author: Hao Luo <haoluo@google.com> > > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > encodes kernel global variables into BTF so that bpf programs can > > directly access them. If there is no need to access kernel global > > variables, it's perfectly fine to use '--btf_encode_force' to skip > > encoding bad symbols into BTF, or '--skip_encoding_btf_vars' to skip > > encoding all global vars all together. I will add these info into the > > updated error message. > > Also cc bpf folks for attention of this bug. > I've already fixed the message as part of > 2e719cca6672 ("btf_encoder: revamp how per-CPU variables are encoded") > It's currently still in the tmp.libbtf_encoder branch in pahole repo. I'm now running: $ grep BTF=y ../build/s390x-v5.9.0+/.config CONFIG_DEBUG_INFO_BTF=y $ make -j24 CROSS_COMPILE=s390x-linux-gnu- ARCH=s390 O=../build/s390x-v5.9.0+/ To do the last test I wanted before moving it to master. - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 18:14 ` Arnaldo Carvalho de Melo @ 2020-10-20 19:02 ` Arnaldo Carvalho de Melo 0 siblings, 0 replies; 14+ messages in thread From: Arnaldo Carvalho de Melo @ 2020-10-20 19:02 UTC (permalink / raw) To: Andrii Nakryiko Cc: Hao Luo, Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Andrii Nakryiko, Alexei Starovoitov, bpf Em Tue, Oct 20, 2020 at 03:14:59PM -0300, Arnaldo Carvalho de Melo escreveu: > Em Tue, Oct 20, 2020 at 10:10:19AM -0700, Andrii Nakryiko escreveu: > > On Tue, Oct 20, 2020 at 10:05 AM Hao Luo <haoluo@google.com> wrote: > > > Thanks for reporting this and cc'ing me. I forgot to update the error > > > messages when renaming the flags. I will send a patch to fix the error > > > message. > > > > The commit > > > > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > > > Author: Hao Luo <haoluo@google.com> > > > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > > encodes kernel global variables into BTF so that bpf programs can > > > directly access them. If there is no need to access kernel global > > > variables, it's perfectly fine to use '--btf_encode_force' to skip > > > encoding bad symbols into BTF, or '--skip_encoding_btf_vars' to skip > > > encoding all global vars all together. I will add these info into the > > > updated error message. > > > > Also cc bpf folks for attention of this bug. > > > I've already fixed the message as part of > > 2e719cca6672 ("btf_encoder: revamp how per-CPU variables are encoded") > > > It's currently still in the tmp.libbtf_encoder branch in pahole repo. > > I'm now running: > > $ grep BTF=y ../build/s390x-v5.9.0+/.config > CONFIG_DEBUG_INFO_BTF=y > $ make -j24 CROSS_COMPILE=s390x-linux-gnu- ARCH=s390 O=../build/s390x-v5.9.0+/ $ ls -la /home/acme/git/build/s390x-v5.9.0+/.tmp_vmlinux.btf -rwxrwxr-x. 1 acme acme 304592928 Oct 20 15:26 /home/acme/git/build/s390x-v5.9.0+/.tmp_vmlinux.btf $ file /home/acme/git/build/s390x-v5.9.0+/.tmp_vmlinux.btf /home/acme/git/build/s390x-v5.9.0+/.tmp_vmlinux.btf: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, BuildID[sha1]=ed39402fdbd7108c1055baaa61cfc6b0e431901d, with debug_info, not stripped $ pahole -F btf -C list_head /home/acme/git/build/s390x-v5.9.0+/.tmp_vmlinux.btf struct list_head { struct list_head * next; /* 0 8 */ struct list_head * prev; /* 8 8 */ /* size: 16, cachelines: 1, members: 2 */ /* last cacheline: 16 bytes */ }; $ $ readelf -wi /home/acme/git/build/s390x-v5.9.0+/vmlinux | grep -m2 DW_AT_producer <28> DW_AT_producer : (indirect string, offset: 0x51): GNU AS 2.34 <3b> DW_AT_producer : (indirect string, offset: 0xeb46): GNU C89 9.2.1 20190827 (Red Hat Cross 9.2.1-3) -m64 -mwarn-dynamicstack -mbackchain -msoft-float -march=z196 -mtune=z196 -mpacked-stack -mindirect-branch=thunk -mfunction-return=thunk -mindirect-branch-table -mrecord-mcount -mnop-mcount -mfentry -mzarch -g -O2 -std=gnu90 -p -fno-strict-aliasing -fno-common -fshort-wchar -fPIE -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks -fno-reorder-blocks -fno-ipa-cp-clone -fno-partial-inlining -fno-stack-protector -fno-var-tracking-assignments -fno-inline-functions-called-once -falign-functions=32 -fno-strict-overflow -fstack-check=no -fconserve-stack -fno-function-sections -fno-data-sections -fsanitize=kernel-address -fasan-shadow-offset=0x18000000000000 -fsanitize=bounds -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=unreachable -fsanitize=signed-integer-overflow -fsanitize=object-size -fsanitize=bool -fsanitize=enum -fsanitize-undefined-trap-on-error -fsanitize-coverage=trace-pc -fsanitize-coverage=trace-cmp --param allow-store-data-races=0 --param asan-globals=1 --param asan-instrumentation-with-call-threshold=0 --param asan-stack=1 --param asan-instrument-allocas=1 $ $ file /home/acme/git/build/s390x-v5.9.0+/vmlinux /home/acme/git/build/s390x-v5.9.0+/vmlinux: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, BuildID[sha1]=fbb252d8dccc11d8e66d6f248d06bcdca4e7db7a, with debug_info, not stripped $ But I noticed that 'btfdiff' is showing differences from output generated from DWARF and the one generated from BTF, the first issue is: [acme@five pahole]$ btfdiff /home/acme/git/build/v5.9.0+/vmlinux <SNIP> @@ -115549,7 +120436,7 @@ struct irq_router_handler { /* XXX 6 bytes hole, try to pack */ - int (*probe)(struct irq_router * , struct pci_dev * , u16 ); /* 8 8 */ + int (*probe)(struct irq_router *, struct pci_dev *, u16); /* 8 8 */ /* size: 16, cachelines: 1, members: 2 */ /* sum members: 10, holes: 1, sum holes: 6 */ [acme@five pahole]$ The BTF output (the one starting with '+' in the diff output) is better, just different than it was before, I'll fix the DWARF one to avoid that needless space for arg lists without names. The other problem I noticed is a bit more worrying: @@ -52,13 +52,29 @@ struct file_system_type { /* last cacheline: 8 bytes */ }; struct qspinlock { - union ; /* 0 4 */ + union { + atomic_t val; /* 0 4 */ + struct { + u8 locked; /* 0 1 */ + u8 pending; /* 1 1 */ + }; /* 0 2 */ + struct { + u16 locked_pending; /* 0 2 */ + u16 tail; /* 2 2 */ + }; /* 0 4 */ + }; /* 0 4 */ /* size: 4, cachelines: 1, members: 1 */ /* last cacheline: 4 bytes */ }; struct qrwlock { - union ; /* 0 4 */ + union { + atomic_t cnts; /* 0 4 */ + struct { + u8 wlocked; /* 0 1 */ + u8 __lstate[3]; /* 1 3 */ + }; /* 0 4 */ + }; /* 0 4 */ arch_spinlock_t wait_lock; /* 4 4 */ /* size: 8, cachelines: 1, members: 2 */ But again, its the DWARF code that is wrong :-) So, for what is being tested here, which is BTF generation, things looks Ok: i.e. using BTF: [acme@five perf]$ pahole qspinlock struct qspinlock { union { atomic_t val; /* 0 4 */ struct { u8 locked; /* 0 1 */ u8 pending; /* 1 1 */ }; /* 0 2 */ struct { u16 locked_pending; /* 0 2 */ u16 tail; /* 2 2 */ }; /* 0 4 */ }; /* 0 4 */ /* size: 4, cachelines: 1, members: 1 */ /* last cacheline: 4 bytes */ }; [acme@five perf]$ While using DWARF: [acme@five perf]$ pahole -F dwarf -C qspinlock struct qspinlock { union ; /* 0 4 */ /* size: 4, cachelines: 1, members: 1 */ /* last cacheline: 4 bytes */ }; [acme@five perf]$ typedef struct qspinlock { union { atomic_t val; /* * By using the whole 2nd least significant byte for the * pending bit, we can allow better optimization of the lock * acquisition for the pending bit holder. */ #ifdef __LITTLE_ENDIAN struct { u8 locked; u8 pending; }; struct { u16 locked_pending; u16 tail; }; #else struct { u16 tail; u16 locked_pending; }; struct { u8 reserved[2]; u8 pending; u8 locked; }; #endif }; } arch_spinlock_t; This is just a heads up, will investigate further... - Arnaldo > To do the last test I wanted before moving it to master. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 12:20 ` Arnaldo Carvalho de Melo 2020-10-20 17:04 ` Hao Luo @ 2020-10-21 6:22 ` Jiri Slaby 2020-10-21 11:29 ` Arnaldo Carvalho de Melo 1 sibling, 1 reply; 14+ messages in thread From: Jiri Slaby @ 2020-10-21 6:22 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Érico Rolim, dwarves, linux-kernel, Arnaldo Carvalho de Melo, Hao Luo, Andrii Nakryiko On 20. 10. 20, 14:20, Arnaldo Carvalho de Melo wrote: >> Yeah, I observe the very same. I reported it at: >> https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > Would it be possible to try with > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > ? Yes, that branch fixes the crashes and the kernel build finishes. The zero-sized symbol error remains. So what should distributions do now -- should we switch to a pahole snapshot for a while? thanks, -- js ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-21 6:22 ` Jiri Slaby @ 2020-10-21 11:29 ` Arnaldo Carvalho de Melo 2020-10-21 15:53 ` Andrii Nakryiko 0 siblings, 1 reply; 14+ messages in thread From: Arnaldo Carvalho de Melo @ 2020-10-21 11:29 UTC (permalink / raw) To: Jiri Slaby Cc: Érico Rolim, dwarves, linux-kernel, Arnaldo Carvalho de Melo, Hao Luo, Andrii Nakryiko Em Wed, Oct 21, 2020 at 08:22:40AM +0200, Jiri Slaby escreveu: > On 20. 10. 20, 14:20, Arnaldo Carvalho de Melo wrote: > > > Yeah, I observe the very same. I reported it at: > > > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > Would it be possible to try with > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > > ? > Yes, that branch fixes the crashes and the kernel build finishes. The > zero-sized symbol error remains. > So what should distributions do now -- should we switch to a pahole snapshot > for a while? That would do the trick, I just completed my testing and pushed to the master branch on kernel.org and github, tests detailed at: https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?id=040fd7f585c9b9fcf4475d294b3f5ddf78405297 There are some minor bug reports I want to address but my ETA right now is the end of this week to release v1.19. - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-21 11:29 ` Arnaldo Carvalho de Melo @ 2020-10-21 15:53 ` Andrii Nakryiko 2020-10-21 16:26 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 14+ messages in thread From: Andrii Nakryiko @ 2020-10-21 15:53 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Hao Luo, Andrii Nakryiko On Wed, Oct 21, 2020 at 6:48 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote: > > Em Wed, Oct 21, 2020 at 08:22:40AM +0200, Jiri Slaby escreveu: > > On 20. 10. 20, 14:20, Arnaldo Carvalho de Melo wrote: > > > > Yeah, I observe the very same. I reported it at: > > > > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > > > Would it be possible to try with > > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > > > ? > > > Yes, that branch fixes the crashes and the kernel build finishes. The > > zero-sized symbol error remains. > > > So what should distributions do now -- should we switch to a pahole snapshot > > for a while? > > That would do the trick, I just completed my testing and pushed to the > master branch on kernel.org and github, tests detailed at: > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?id=040fd7f585c9b9fcf4475d294b3f5ddf78405297 > > There are some minor bug reports I want to address but my ETA right now > is the end of this week to release v1.19. I've just sent a patch that skips zero-sized ELF symbols without a warning or error. > > - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-21 15:53 ` Andrii Nakryiko @ 2020-10-21 16:26 ` Arnaldo Carvalho de Melo 0 siblings, 0 replies; 14+ messages in thread From: Arnaldo Carvalho de Melo @ 2020-10-21 16:26 UTC (permalink / raw) To: Andrii Nakryiko Cc: Jiri Slaby, Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Hao Luo, Andrii Nakryiko Em Wed, Oct 21, 2020 at 08:53:16AM -0700, Andrii Nakryiko escreveu: > On Wed, Oct 21, 2020 at 6:48 AM Arnaldo Carvalho de Melo > <acme@kernel.org> wrote: > > > > Em Wed, Oct 21, 2020 at 08:22:40AM +0200, Jiri Slaby escreveu: > > > On 20. 10. 20, 14:20, Arnaldo Carvalho de Melo wrote: > > > > > Yeah, I observe the very same. I reported it at: > > > > > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > > > > > Would it be possible to try with > > > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=tmp.libbtf_encoder > > > > ? > > > > > Yes, that branch fixes the crashes and the kernel build finishes. The > > > zero-sized symbol error remains. > > > > > So what should distributions do now -- should we switch to a pahole snapshot > > > for a while? > > > > That would do the trick, I just completed my testing and pushed to the > > master branch on kernel.org and github, tests detailed at: > > > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?id=040fd7f585c9b9fcf4475d294b3f5ddf78405297 > > > > There are some minor bug reports I want to address but my ETA right now > > is the end of this week to release v1.19. > > I've just sent a patch that skips zero-sized ELF symbols without a > warning or error. Got it, applied, Jiri, please consider testing it and providing a Tested-by, in addition to the Reported-by that I'm adding now. Thanks! - Arnaldo ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 9:01 ` Jiri Slaby 2020-10-20 12:20 ` Arnaldo Carvalho de Melo @ 2020-10-20 17:15 ` Andrii Nakryiko 2020-10-21 5:52 ` Jiri Slaby 1 sibling, 1 reply; 14+ messages in thread From: Andrii Nakryiko @ 2020-10-20 17:15 UTC (permalink / raw) To: Jiri Slaby Cc: Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Hao Luo On Tue, Oct 20, 2020 at 3:51 AM Jiri Slaby <jirislaby@kernel.org> wrote: > > Hi, > > On 19. 10. 20, 1:18, Érico Rolim wrote: > > I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has > > `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole > > version 1.18 segfaults during the build, as can be seen below: > > > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: > > '__kvm_nvhe_arm64_ssbd_callback_required', cu: > > 'arch/arm64/kernel/cpu_errata.c'). > > The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: > __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; What's readelf's output for that symbol? If it's legal for SST_OBJECT to have size zero, then we should just skip those in pahole. But it shouldn't crash in either case, of course. But as Arnaldo mentioned, that code changed significantly recently, so please check with latest pahole from tmp.libbtf_encoder branch. > > > PAHOLE: Error: Use '-j' or '--force' to ignore such symbols and force > > emit the btf. > > scripts/link-vmlinux.sh: line 141: 43837 Segmentation fault > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > LD .tmp_vmlinux.kallsyms1 > > KSYM .tmp_vmlinux.kallsyms1.o > > LD .tmp_vmlinux.kallsyms2 > > KSYM .tmp_vmlinux.kallsyms2.o > > LD vmlinux > > BTFIDS vmlinux > > FAILED: load BTF from vmlinux: Unknown error -2make: *** > > [Makefile:1162: vmlinux] Error 255 > > > > It is possible to force the build to continue if > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > in scripts/link-vmlinux.sh is changed to > > > > LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J --btf_encode_force ${1} > > > > The suggested `-j` or `--force` flags don't exist, since they were removed in > > [1]. I believe `--btf_encode_force` should be suggested instead. > > Agreed, '--btf_encode_force' makes pahole to proceed without crashes. > > > It should be noted that the same build, but with pahole version 1.17, works > > without issue, so I think this is either a regression in pahole or the script > > will need to be changed for newer versions of pahole. > > Yeah, I observe the very same. I reported it at: > https://bugzilla.suse.com/show_bug.cgi?id=1177921 > > The backtrace: [...] > > > I suspect: > commit f3d9054ba8ff1df0fc44e507e3a01c0964cabd42 > Author: Hao Luo <haoluo@google.com> > AuthorDate: Wed Jul 8 13:44:10 2020 -0700 > > btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. > > > Which added this machinery (btf_elf__add_datasec_type in particular). > > > - [1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/pahole.c?h=v1.18&id=1abc001417b579b86a9b27ff88c9095d8f498a46 > > > > Thanks, > > Érico > > > > > -- > js > suse labs ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 2020-10-20 17:15 ` Andrii Nakryiko @ 2020-10-21 5:52 ` Jiri Slaby 0 siblings, 0 replies; 14+ messages in thread From: Jiri Slaby @ 2020-10-21 5:52 UTC (permalink / raw) To: Andrii Nakryiko Cc: Érico Rolim, dwarves, open list, Arnaldo Carvalho de Melo, Hao Luo On 20. 10. 20, 19:15, Andrii Nakryiko wrote: > On Tue, Oct 20, 2020 at 3:51 AM Jiri Slaby <jirislaby@kernel.org> wrote: >> >> Hi, >> >> On 19. 10. 20, 1:18, Érico Rolim wrote: >>> I'm trying to build kernel 5.9.1 for arm64, and my dotconfig has >>> `CONFIG_DEBUG_INFO_BTF=y`, which requires pahole for building. However, pahole >>> version 1.18 segfaults during the build, as can be seen below: >>> >>> PAHOLE: Error: Found symbol of zero size when encoding btf (sym: >>> '__kvm_nvhe_arm64_ssbd_callback_required', cu: >>> 'arch/arm64/kernel/cpu_errata.c'). >> >> The symbol is an alias coming from arch/arm64/kernel/vmlinux.lds: >> __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;; > > What's readelf's output for that symbol? If it's legal for SST_OBJECT > to have size zero, then we should just skip those in pahole. But it > shouldn't crash in either case, of course. But as Arnaldo mentioned, > that code changed significantly recently, so please check with latest > pahole from tmp.libbtf_encoder branch. ... >> Yeah, I observe the very same. I reported it at: >> https://bugzilla.suse.com/show_bug.cgi?id=1177921 If you looked here, you would see: > $ readelf -Ws vml |grep arm64_ssbd_callback_re > 154271: ffff80001133e000 0 OBJECT GLOBAL DEFAULT 22 __kvm_nvhe_arm64_ssbd_callback_required > 159609: ffff80001133e000 8 OBJECT WEAK DEFAULT 22 arm64_ssbd_callback_required Yes, its zero-sized. And yes, the error happens even with tmp.libbtf_encoder, but pahole doesn't crash and the build finishes fine. thanks, -- js ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-10-21 16:26 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-10-18 23:18 Segfault in pahole 1.18 when building kernel 5.9.1 for arm64 Érico Rolim 2020-10-20 9:01 ` Jiri Slaby 2020-10-20 12:20 ` Arnaldo Carvalho de Melo 2020-10-20 17:04 ` Hao Luo 2020-10-20 17:10 ` Andrii Nakryiko 2020-10-20 17:18 ` Hao Luo 2020-10-20 18:14 ` Arnaldo Carvalho de Melo 2020-10-20 19:02 ` Arnaldo Carvalho de Melo 2020-10-21 6:22 ` Jiri Slaby 2020-10-21 11:29 ` Arnaldo Carvalho de Melo 2020-10-21 15:53 ` Andrii Nakryiko 2020-10-21 16:26 ` Arnaldo Carvalho de Melo 2020-10-20 17:15 ` Andrii Nakryiko 2020-10-21 5:52 ` Jiri Slaby
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).