All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alexandre Ferreira <alexandref75@gmail.com>
To: Jiri Olsa <olsajiri@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>, Daniel Xu <dxu@dxuuu.xyz>,
	"bpf@vger.kernel.org" <bpf@vger.kernel.org>,
	Arnaldo Carvalho de Melo <acme@kernel.org>
Subject: Re: Kernel build fail with 'btf_encoder__encode: btf__dedup failed!'
Date: Fri, 10 Feb 2023 09:37:27 -0600	[thread overview]
Message-ID: <CAMHq1ZuhecqT-YX8+yzUyhx8tmnQ7CDyuXV4GOuG=z44XTFsaw@mail.gmail.com> (raw)
In-Reply-To: <Y+ZWAesOAY5wjG6i@krava>

Jiri,

On Fri, Feb 10, 2023 at 8:34 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Fri, Feb 10, 2023 at 08:02:23AM -0600, Alexandre Peixoto Ferreira wrote:
> > Alam,
> >
> > On 2/9/23 07:07, Alan Maguire wrote:
> > > On 09/02/2023 04:15, Alexandre Peixoto Ferreira wrote:
> > > > Jiri,
> > > >
> > > > On 1/31/23 09:18, Jiri Olsa wrote:
> > > > > On Sat, Jan 28, 2023 at 01:23:25PM -0600, Alexandre Peixoto Ferreira wrote:
> > > > > > Jirka and Daniel,
> > > > > >
> > > > > > On 1/27/23 18:00, Jiri Olsa wrote:
> > > > > > > On Fri, Jan 27, 2023 at 04:28:54PM -0600, Alexandre Peixoto Ferreira wrote:
> > > > > > > > On 1/24/23 00:13, Daniel Xu wrote:
> > > > > > > > > Hi Jiri,
> > > > > > > > >
> > > > > > > > > On Mon, Jan 23, 2023, at 1:06 AM, Jiri Olsa wrote:
> > > > > > > > > > On Sun, Jan 22, 2023 at 10:48:44AM -0700, Daniel Xu wrote:
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > I'm getting the following error during build:
> > > > > > > > > > >
> > > > > > > > > > >             $ ./tools/testing/selftests/bpf/vmtest.sh -j30
> > > > > > > > > > >             [...]
> > > > > > > > > > >               BTF     .btf.vmlinux.bin.o
> > > > > > > > > > >             btf_encoder__encode: btf__dedup failed!
> > > > > > > > > > >             Failed to encode BTF
> > > > > > > > > > >               LD      .tmp_vmlinux.kallsyms1
> > > > > > > > > > >               NM      .tmp_vmlinux.kallsyms1.syms
> > > > > > > > > > >               KSYMS   .tmp_vmlinux.kallsyms1.S
> > > > > > > > > > >               AS      .tmp_vmlinux.kallsyms1.S
> > > > > > > > > > >               LD      .tmp_vmlinux.kallsyms2
> > > > > > > > > > >               NM      .tmp_vmlinux.kallsyms2.syms
> > > > > > > > > > >               KSYMS   .tmp_vmlinux.kallsyms2.S
> > > > > > > > > > >               AS      .tmp_vmlinux.kallsyms2.S
> > > > > > > > > > >               LD      .tmp_vmlinux.kallsyms3
> > > > > > > > > > >               NM      .tmp_vmlinux.kallsyms3.syms
> > > > > > > > > > >               KSYMS   .tmp_vmlinux.kallsyms3.S
> > > > > > > > > > >               AS      .tmp_vmlinux.kallsyms3.S
> > > > > > > > > > >               LD      vmlinux
> > > > > > > > > > >               BTFIDS  vmlinux
> > > > > > > > > > >             FAILED: load BTF from vmlinux: No such file or directory
> > > > > > > > > > >             make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 255
> > > > > > > > > > >             make[1]: *** Deleting file 'vmlinux'
> > > > > > > > > > >             make: *** [Makefile:1264: vmlinux] Error 2
> > > > > > > > > > >
> > > > > > > > > > > This happens on both bpf-next/master (84150795a49) and 6.2-rc5
> > > > > > > > > > > (2241ab53cb).
> > > > > > > > > > >
> > > > > > > > > > > I've also tried arch linux pahole 1:1.24+r29+g02d67c5-1 as well as
> > > > > > > > > > > upstream pahole on master (02d67c5176) and upstream pahole on
> > > > > > > > > > > next (2ca56f4c6f659).
> > > > > > > > > > >
> > > > > > > > > > > Of the above 6 combinations, I think I've tried all of them (maybe
> > > > > > > > > > > missing 1 or 2).
> > > > > > > > > > >
> > > > > > > > > > > Looks like GCC got updated recently on my machine, so perhaps
> > > > > > > > > > > it's related?
> > > > > > > > > > >
> > > > > > > > > > >             CONFIG_CC_VERSION_TEXT="gcc (GCC) 12.2.1 20230111"
> > > > > > > > > > >
> > > > > > > > > > > I'll try some debugging, but just wanted to report it first.
> > > > > > > > > > hi,
> > > > > > > > > > I can't reproduce that.. can you reproduce it outside vmtest.sh?
> > > > > > > > > >
> > > > > > > > > > there will be lot of output with patch below, but could contain
> > > > > > > > > > some more error output
> > > > > > > > > Thanks for the hints. Doing a regular build outside of vmtest.sh
> > > > > > > > > seems to work ok. So maybe it's a difference in the build config.
> > > > > > > > >
> > > > > > > > > I'll put a little more time into debugging to see if it goes anywhere.
> > > > > > > > > But I'll have to get back to the regularly scheduled programming
> > > > > > > > > soon.
> > > > > > > > 6.2-rc5 compiles correctly when CONFIG_X86_KERNEL_IBT is commented but fails
> > > > > > > > in pahole when CONFIG_X86_KERNEL_IBT is set.
> > > > > > > could you plese attach your config and the build error?
> > > > > > > I can't reproduce that
> > > > > > >
> > > > > > > thanks,
> > > > > > > jirka
> > > > > > My working .config is available at https://pastebin.pl/view/bef3765c
> > > > > > change CONFIG_X86_KERNEL_IBT to y to get the error.
> > > > > >
> > > > > > The error is similar to Daniel's and is shown below:
> > > > > >
> > > > > >     LD      .tmp_vmlinux.btf
> > > > > >     BTF     .btf.vmlinux.bin.o
> > > > > > btf_encoder__encode: btf__dedup failed!
> > > > > > Failed to encode BTF
> > > > > >     LD      .tmp_vmlinux.kallsyms1
> > > > > >     NM      .tmp_vmlinux.kallsyms1.syms
> > > > > >     KSYMS   .tmp_vmlinux.kallsyms1.S
> > > > > >     AS      .tmp_vmlinux.kallsyms1.S
> > > > > >     LD      .tmp_vmlinux.kallsyms2
> > > > > >     NM      .tmp_vmlinux.kallsyms2.syms
> > > > > >     KSYMS   .tmp_vmlinux.kallsyms2.S
> > > > > >     AS      .tmp_vmlinux.kallsyms2.S
> > > > > >     LD      .tmp_vmlinux.kallsyms3
> > > > > >     NM      .tmp_vmlinux.kallsyms3.syms
> > > > > >     KSYMS   .tmp_vmlinux.kallsyms3.S
> > > > > >     AS      .tmp_vmlinux.kallsyms3.S
> > > > > >     LD      vmlinux
> > > > > >     BTFIDS  vmlinux
> > > > > > FAILED: load BTF from vmlinux: No such file or directory
> > > > > > make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 255
> > > > > > make[1]: *** Deleting file 'vmlinux'
> > > > > > make: *** [Makefile:1264: vmlinux] Error 2
> > > > > I can't reproduce that.. I tried with gcc versions:
> > > > >
> > > > >     gcc (GCC) 13.0.1 20230117 (Red Hat 13.0.1-0)
> > > > >     gcc (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4)
> > > > >
> > > > > I haven't found fedora setup with 12.2.1 20230111 yet
> > > > >
> > > > > I tried alsa with latest pahole master branch
> > > > >
> > > > > were you guys able to get any more verbose output
> > > > > that I suggested earlier?
> > > > >
> > > > > jirka
> > > > I compiled with and without IBT using the -V on pahole (LLVM_OBJCOPY=objcopy pahole -V -J --btf_gen_floats -j .tmp_vmlinux.btf) and the outfiles are a little too big (540MB). The error happens with this CONST type pointing to itself. That does not happen with the IBT option removed.
> > > >
> > > > $ grep  -n "CONST (anon) type_id" /tmp/with_IBT  | more
> > > > 346:[2] CONST (anon) type_id=2
> > > > 349:[5] CONST (anon) type_id=5
> > > > 351:[7] CONST (anon) type_id=7
> > > > 356:[12] CONST (anon) type_id=12
> > > > 363:[19] CONST (anon) type_id=19
> > > > 373:[29] CONST (anon) type_id=29
> > > > 375:[31] CONST (anon) type_id=31
> > > > 409:[63] CONST (anon) type_id=63
> > > > 444:[89] CONST (anon) type_id=0
> > > > 472:[97] CONST (anon) type_id=97
> > > > 616:[129] CONST (anon) type_id=129
> > > > 652:[131] CONST (anon) type_id=131
> > > > 1319:[234] CONST (anon) type_id=234
> > > > 1372:[246] CONST (anon) type_id=246
> > > > ....
> > > >
> > > > $diff -ru with_IBT without_IBT
> > > > --- with_IBT 2023-01-31 09:39:24.915912735 -0600
> > > > +++ without_IBT 2023-01-31 09:46:23.456005278 -0600
> > > > @@ -340,346 +340,14800 @@
> > > >   Found per-CPU symbol 'cpu_tlbstate_shared' at address 0x2c040
> > > >   Found per-CPU symbol 'mce_poll_banks' at address 0x1ad20
> > > >   Found 341 per-CPU variables!
> > > > -Found 61470 functions!
> > > > +Found 61462 functions!
> > > > +File .tmp_vmlinux.btf:
> > > > +[1] FUNC_PROTO (anon) return=0 args=(void)
> > > > +[2] FUNC verify_cpu type_id=1
> > > > +[3] FUNC_PROTO (anon) return=0 args=(void)
> > > > +[4] FUNC sev_verify_cbit type_id=3
> > > > +search cu 'arch/x86/kernel/head_64.S' for percpu global variables.
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > +Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > > +Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > > +Found per-CPU symbol 'cpu_kick_mask' at address 0x19f78
> > > > +Found per-CPU symbol 'cpu_tsc_khz' at address 0x19f88
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > +Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > > +Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > > +Found per-CPU symbol 'current_tsc_ratio' at address 0x19fa0
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > +Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > > +Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > > +Found per-CPU symbol 'cpu_kick_mask' at address 0x19f78
> > > > +Found per-CPU symbol 'cpu_kick_mask' at address 0x19f78
> > > > +Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > > +Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > +Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > > +Found per-CPU symbol 'cpu_tsc_khz' at address 0x19f88
> > > > +Found per-CPU symbol 'last_nmi_rip' at address 0x1a018
> > > > +Found per-CPU symbol 'nmi_stats' at address 0x1a030
> > > > +Found per-CPU symbol 'swallow_nmi' at address 0x1a020
> > > > +Found per-CPU symbol 'nmi_state' at address 0x1a010
> > > > +Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > > +Found per-CPU symbol 'nmi_cr2' at address 0x1a008
> > > > +Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > > +Found per-CPU symbol 'cpu_tsc_khz' at address 0x19f88
> > > > +Found per-CPU symbol 'cpu_kick_mask' at address 0x19f78
> > > > +Found per-CPU symbol 'cpu_kick_mask' at address 0x19f78
> > > > +Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > > ...
> > > >
> > > > And the lines 342-365 of the with_IBT result:
> > > >       342 Found 341 per-CPU variables!
> > > >       343 Found 61470 functions!
> > > >       344 File .tmp_vmlinux.btf:
> > > >       345 [1] INT long unsigned int size=8 nr_bits=64 encoding=(none)
> > > >       346 [2] CONST (anon) type_id=2
> > > >       347 [3] PTR (anon) type_id=6
> > > >       348 [4] INT char size=1 nr_bits=8 encoding=(none)
> > > >       349 [5] CONST (anon) type_id=5
> > > >       350 [6] INT unsigned int size=4 nr_bits=32 encoding=(none)
> > > >       351 [7] CONST (anon) type_id=7
> > > >       352 [8] TYPEDEF __s8 type_id=10
> > > >       353 [9] INT signed char size=1 nr_bits=8 encoding=SIGNED
> > > >       354 [10] TYPEDEF __u8 type_id=12
> > > >       355 [11] INT unsigned char size=1 nr_bits=8 encoding=(none)
> > > >       356 [12] CONST (anon) type_id=12
> > > >       357 [13] TYPEDEF __s16 type_id=15
> > > >       358 [14] INT short int size=2 nr_bits=16 encoding=SIGNED
> > > >       359 [15] TYPEDEF __u16 type_id=17
> > > >       360 [16] INT short unsigned int size=2 nr_bits=16 encoding=(none)
> > > >       361 [17] TYPEDEF __s32 type_id=19
> > > >       362 [18] INT int size=4 nr_bits=32 encoding=SIGNED
> > > >       363 [19] CONST (anon) type_id=19
> > > >       364 [20] TYPEDEF __u32 type_id=7
> > > >       365 [21] TYPEDEF __s64 type_id=23
> > > >
> > > > lines 342-362 of without_IBT
> > > >
> > > >       342 Found 341 per-CPU variables!
> > > >       343 Found 61462 functions!
> > > >       344 File .tmp_vmlinux.btf:
> > > >       345 [1] FUNC_PROTO (anon) return=0 args=(void)
> > > >       346 [2] FUNC verify_cpu type_id=1
> > > >       347 [3] FUNC_PROTO (anon) return=0 args=(void)
> > > >       348 [4] FUNC sev_verify_cbit type_id=3
> > > >       349 search cu 'arch/x86/kernel/head_64.S' for percpu global variables.
> > > >       350 Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > >       351 Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > >       352 Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > >       353 Found per-CPU symbol 'cpu_kick_mask' at address 0x19f78
> > > >       354 Found per-CPU symbol 'cpu_tsc_khz' at address 0x19f88
> > > >       355 Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > >       356 Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > >       357 Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > >       358 Found per-CPU symbol 'perf_nmi_tstamp' at address 0x19f70
> > > >       359 Found per-CPU symbol 'current_tsc_ratio' at address 0x19fa0
> > > >       360 Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > >       361 Found per-CPU symbol 'cpu_loops_per_jiffy' at address 0x18a08
> > > >       362 Found per-CPU symbol 'kvm_running_vcpu' at address 0x19f80
> > > >
> > > > If the full debug files are useful or a target grep or diff is better let me know.
> > > >
> > > I managed to reproduce this too with IBT enabled; one thing I
> > > noticed is with pahole built with an up-to-date libbpf and the
> > > changes in https://github.com/acmel/dwarves/tree/next, the problem
> > > went away. I didn't have time to root-cause it yet however.
> > >
> > > Not sure if you're in a position to do this, but if you can,
> > > would you mind building pahole from
> > >
> > > https://github.com/acmel/dwarves/tree/next
> > >
> > > ...and re-testing to see if that helps? Thanks!
> > >
> > > Alan
> > > > Thanks,
> > > >
> > I tried with libbpf compiled from master
> > https://github.com/libbpf/libbpf.git and pahole compiled from next branch on
> > https://github.com/acmel/dwarve with the same result.
> > With IBT enabled pahole fails and removing it results in a successful
> > kernel.
>
> hi,
> in case it slipped, you also need to add new options for pahole:
>   https://lore.kernel.org/bpf/1675949331-27935-1-git-send-email-alan.maguire@oracle.com/
>
> should be added for version 124 for now
>
> jirka


Added the patch to include options on pahole but same problem.
$ pahole --version
v1.25
$ ls -l /usr/lib64/libbpf.so.1.2.0
-rwxr-xr-x 1 root root 422088 Feb  9 13:23 /usr/lib64/libbpf.so.1.2.0

  UPD     include/generated/utsversion.h
  CC      init/version-timestamp.o
  LD      .tmp_vmlinux.btf
  BTF     .btf.vmlinux.bin.o
LLVM_OBJCOPY=objcopy pahole -J --btf_gen_floats -j
--skip_encoding_btf_inconsistent_proto --btf_gen_optimized
.tmp_vmlinux.btf
btf_encoder__encode: btf__dedup failed!
Failed to encode BTF

Thanks,

  reply	other threads:[~2023-02-10 15:37 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-22 17:48 Kernel build fail with 'btf_encoder__encode: btf__dedup failed!' Daniel Xu
2023-01-22 21:25 ` Daniel Xu
2023-01-23  8:06 ` Jiri Olsa
2023-01-24  6:13   ` Daniel Xu
2023-01-27 22:28     ` Alexandre Peixoto Ferreira
2023-01-28  0:00       ` Jiri Olsa
2023-01-28  0:21         ` Daniel Xu
2023-01-28  0:23           ` Daniel Xu
2023-01-28 19:23         ` Alexandre Peixoto Ferreira
2023-01-31 15:18           ` Jiri Olsa
2023-02-09  0:21             ` Vicki Pfau
2023-02-09  4:15             ` Alexandre Peixoto Ferreira
2023-02-09 13:07               ` Alan Maguire
2023-02-10 14:02                 ` Alexandre Peixoto Ferreira
2023-02-10 14:34                   ` Jiri Olsa
2023-02-10 15:37                     ` Alexandre Ferreira [this message]
2023-02-14  6:33                       ` Tianyi Liu
2023-04-11 17:00                         ` [PATCH] vmlinux.lds.h: Force-align ELF notes section to four bytes Josh Poimboeuf
2023-04-12  7:10                           ` Tianyi Liu
2023-04-12 16:30                             ` Josh Poimboeuf
2023-04-13  2:21                               ` Joan Bruguera Micó
2023-04-13  9:23                                 ` Tianyi Liu
2023-04-13 18:59                                   ` [PATCH] vmlinux.lds.h: Discard .note.gnu.property section Josh Poimboeuf
2023-04-16 19:02                                     ` Joan Bruguera Micó
2023-04-18 21:47                                       ` Josh Poimboeuf
2023-04-18 21:49                                         ` [PATCH v2] " Josh Poimboeuf
2023-04-13  4:54                               ` [PATCH] vmlinux.lds.h: Force-align ELF notes section to four bytes Tianyi Liu
2023-04-16 18:52                                 ` Joan Bruguera Micó
2023-04-27  5:14                                   ` Joan Bruguera Micó
2023-05-18 11:00 ` [tip: objtool/urgent] vmlinux.lds.h: Discard .note.gnu.property section tip-bot2 for Josh Poimboeuf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAMHq1ZuhecqT-YX8+yzUyhx8tmnQ7CDyuXV4GOuG=z44XTFsaw@mail.gmail.com' \
    --to=alexandref75@gmail.com \
    --cc=acme@kernel.org \
    --cc=alan.maguire@oracle.com \
    --cc=bpf@vger.kernel.org \
    --cc=dxu@dxuuu.xyz \
    --cc=olsajiri@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.