* Kernel build error on BTFIDS vmlinux
@ 2020-08-18 8:55 Jesper Dangaard Brouer
2020-08-18 9:14 ` Jiri Olsa
0 siblings, 1 reply; 10+ messages in thread
From: Jesper Dangaard Brouer @ 2020-08-18 8:55 UTC (permalink / raw)
To: Jiri Olsa; +Cc: brouer, linux-kernel, netdev, sdf, andriin
On latest DaveM net-git tree (06a4ec1d9dc652), after linking (LD vmlinux) the
"BTFIDS vmlinux" fails. Are anybody else experiencing this? Are there already a
fix? (just returned from vacation so not fully up-to-date on ML yet)
The tool which is called and error message:
./tools/bpf/resolve_btfids/resolve_btfids vmlinux
FAILED elf_update(WRITE): invalid section alignment
Note, the tool is only called when CONFIG_DEBUG_INFO_BTF is enabled.
I saved a copy of vmlinux and ran the tool manually with verbose
options, the output is provided below signature.
- -
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
$ ./tools/bpf/resolve_btfids/resolve_btfids -vv vmlinux.err.bak
section(1) .text, size 12588824, link 0, flags 6, type=1
section(2) .rodata, size 4424758, link 0, flags 3, type=1
section(3) .pci_fixup, size 12736, link 0, flags 2, type=1
section(4) __ksymtab, size 58620, link 0, flags 2, type=1
section(5) __ksymtab_gpl, size 56592, link 0, flags 2, type=1
section(6) __kcrctab, size 19540, link 0, flags 2, type=1
section(7) __kcrctab_gpl, size 18864, link 0, flags 2, type=1
section(8) __ksymtab_strings, size 180372, link 0, flags 32, type=1
section(9) __param, size 14000, link 0, flags 2, type=1
section(10) __modver, size 152, link 0, flags 2, type=1
section(11) __ex_table, size 21864, link 0, flags 2, type=1
section(12) .notes, size 60, link 0, flags 2, type=7
section(13) .BTF, size 3345350, link 0, flags 2, type=1
section(14) .BTF_ids, size 100, link 0, flags 2, type=1
section(15) .data, size 2243456, link 0, flags 3, type=1
section(16) __bug_table, size 87804, link 0, flags 3, type=1
section(17) .orc_unwind_ip, size 1625580, link 0, flags 2, type=1
section(18) .orc_unwind, size 2438370, link 0, flags 2, type=1
section(19) .orc_lookup, size 196708, link 0, flags 3, type=8
section(20) .vvar, size 4096, link 0, flags 3, type=1
section(21) .data..percpu, size 178840, link 0, flags 3, type=1
section(22) .init.text, size 349579, link 0, flags 6, type=1
section(23) .altinstr_aux, size 3367, link 0, flags 6, type=1
section(24) .init.data, size 1584032, link 0, flags 3, type=1
section(25) .x86_cpu_dev.init, size 24, link 0, flags 2, type=1
section(26) .parainstructions, size 316, link 0, flags 2, type=1
section(27) .altinstructions, size 15015, link 0, flags 2, type=1
section(28) .altinstr_replacement, size 3756, link 0, flags 6, type=1
section(29) .iommu_table, size 160, link 0, flags 2, type=1
section(30) .apicdrivers, size 32, link 0, flags 3, type=1
section(31) .exit.text, size 5195, link 0, flags 6, type=1
section(32) .smp_locks, size 32768, link 0, flags 2, type=1
section(33) .data_nosave, size 0, link 0, flags 1, type=1
section(34) .bss, size 3805184, link 0, flags 3, type=8
section(35) .brk, size 155648, link 0, flags 3, type=8
section(36) .comment, size 44, link 0, flags 30, type=1
section(37) .debug_aranges, size 45684, link 0, flags 800, type=1
section(38) .debug_info, size 129098181, link 0, flags 800, type=1
section(39) .debug_abbrev, size 1152583, link 0, flags 800, type=1
section(40) .debug_line, size 7374522, link 0, flags 800, type=1
section(41) .debug_frame, size 702463, link 0, flags 800, type=1
section(42) .debug_str, size 1017606, link 0, flags 830, type=1
section(43) .debug_loc, size 3019453, link 0, flags 800, type=1
section(44) .debug_ranges, size 1744583, link 0, flags 800, type=1
section(45) .symtab, size 2955888, link 46, flags 0, type=2
section(46) .strtab, size 2613072, link 0, flags 0, type=3
section(47) .shstrtab, size 525, link 0, flags 0, type=3
adding symbol seq_file
adding symbol bpf_map
adding symbol task_struct
adding symbol file
adding symbol bpf_prog
adding symbol bpf_ctx_convert
adding symbol sk_buff
adding symbol xdp_buff
adding symbol inet_sock
adding symbol inet_connection_sock
adding symbol inet_request_sock
adding symbol inet_timewait_sock
adding symbol request_sock
adding symbol sock
adding symbol sock_common
adding symbol tcp_sock
adding symbol tcp_request_sock
adding symbol tcp_timewait_sock
adding symbol tcp6_sock
adding symbol udp_sock
adding symbol udp6_sock
adding symbol netlink_sock
adding symbol fib6_info
patching addr 36: ID 21502 [xdp_buff]
patching addr 84: ID 63192 [udp_sock]
patching addr 88: ID 63195 [udp6_sock]
patching addr 76: ID 66968 [tcp_timewait_sock]
patching addr 68: ID 61353 [tcp_sock]
patching addr 72: ID 61567 [tcp_request_sock]
patching addr 80: ID 63196 [tcp6_sock]
patching addr 12: ID 169 [task_struct]
patching addr 28: ID 169 [task_struct]
patching addr 64: ID 4401 [sock_common]
patching addr 60: ID 2894 [sock]
patching addr 32: ID 3116 [sk_buff]
patching addr 0: ID 1683 [seq_file]
patching addr 4: ID 1683 [seq_file]
patching addr 56: ID 4458 [request_sock]
patching addr 92: ID 65748 [netlink_sock]
patching addr 52: ID 66629 [inet_timewait_sock]
patching addr 40: ID 37652 [inet_sock]
patching addr 48: ID 61566 [inet_request_sock]
patching addr 44: ID 61337 [inet_connection_sock]
patching addr 16: ID 491 [file]
patching addr 96: ID 56653 [fib6_info]
patching addr 20: ID 3099 [bpf_prog]
patching addr 8: ID 1926 [bpf_map]
patching addr 24: ID 21629 [bpf_ctx_convert]
FAILED elf_update(WRITE): invalid section alignment
update failed for vmlinux.err.bak
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index e6e2d9e5ff48..718b2c0ee7ea 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -227,6 +227,7 @@ cleanup()
rm -f .tmp_System.map
rm -f .tmp_vmlinux*
rm -f System.map
+ cp vmlinux vmlinux.err.bak
rm -f vmlinux
rm -f vmlinux.o
}
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-18 8:55 Kernel build error on BTFIDS vmlinux Jesper Dangaard Brouer
@ 2020-08-18 9:14 ` Jiri Olsa
2020-08-18 10:56 ` Jiri Olsa
0 siblings, 1 reply; 10+ messages in thread
From: Jiri Olsa @ 2020-08-18 9:14 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: linux-kernel, netdev, sdf, andriin
On Tue, Aug 18, 2020 at 10:55:55AM +0200, Jesper Dangaard Brouer wrote:
>
> On latest DaveM net-git tree (06a4ec1d9dc652), after linking (LD vmlinux) the
> "BTFIDS vmlinux" fails. Are anybody else experiencing this? Are there already a
> fix? (just returned from vacation so not fully up-to-date on ML yet)
>
> The tool which is called and error message:
> ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> FAILED elf_update(WRITE): invalid section alignment
hi,
could you send your .config as well?
thanks,
jirka
>
> Note, the tool is only called when CONFIG_DEBUG_INFO_BTF is enabled.
>
> I saved a copy of vmlinux and ran the tool manually with verbose
> options, the output is provided below signature.
>
> - -
> Best regards,
> Jesper Dangaard Brouer
> MSc.CS, Principal Kernel Engineer at Red Hat
> LinkedIn: http://www.linkedin.com/in/brouer
>
> $ ./tools/bpf/resolve_btfids/resolve_btfids -vv vmlinux.err.bak
> section(1) .text, size 12588824, link 0, flags 6, type=1
> section(2) .rodata, size 4424758, link 0, flags 3, type=1
> section(3) .pci_fixup, size 12736, link 0, flags 2, type=1
> section(4) __ksymtab, size 58620, link 0, flags 2, type=1
> section(5) __ksymtab_gpl, size 56592, link 0, flags 2, type=1
> section(6) __kcrctab, size 19540, link 0, flags 2, type=1
> section(7) __kcrctab_gpl, size 18864, link 0, flags 2, type=1
> section(8) __ksymtab_strings, size 180372, link 0, flags 32, type=1
> section(9) __param, size 14000, link 0, flags 2, type=1
> section(10) __modver, size 152, link 0, flags 2, type=1
> section(11) __ex_table, size 21864, link 0, flags 2, type=1
> section(12) .notes, size 60, link 0, flags 2, type=7
> section(13) .BTF, size 3345350, link 0, flags 2, type=1
> section(14) .BTF_ids, size 100, link 0, flags 2, type=1
> section(15) .data, size 2243456, link 0, flags 3, type=1
> section(16) __bug_table, size 87804, link 0, flags 3, type=1
> section(17) .orc_unwind_ip, size 1625580, link 0, flags 2, type=1
> section(18) .orc_unwind, size 2438370, link 0, flags 2, type=1
> section(19) .orc_lookup, size 196708, link 0, flags 3, type=8
> section(20) .vvar, size 4096, link 0, flags 3, type=1
> section(21) .data..percpu, size 178840, link 0, flags 3, type=1
> section(22) .init.text, size 349579, link 0, flags 6, type=1
> section(23) .altinstr_aux, size 3367, link 0, flags 6, type=1
> section(24) .init.data, size 1584032, link 0, flags 3, type=1
> section(25) .x86_cpu_dev.init, size 24, link 0, flags 2, type=1
> section(26) .parainstructions, size 316, link 0, flags 2, type=1
> section(27) .altinstructions, size 15015, link 0, flags 2, type=1
> section(28) .altinstr_replacement, size 3756, link 0, flags 6, type=1
> section(29) .iommu_table, size 160, link 0, flags 2, type=1
> section(30) .apicdrivers, size 32, link 0, flags 3, type=1
> section(31) .exit.text, size 5195, link 0, flags 6, type=1
> section(32) .smp_locks, size 32768, link 0, flags 2, type=1
> section(33) .data_nosave, size 0, link 0, flags 1, type=1
> section(34) .bss, size 3805184, link 0, flags 3, type=8
> section(35) .brk, size 155648, link 0, flags 3, type=8
> section(36) .comment, size 44, link 0, flags 30, type=1
> section(37) .debug_aranges, size 45684, link 0, flags 800, type=1
> section(38) .debug_info, size 129098181, link 0, flags 800, type=1
> section(39) .debug_abbrev, size 1152583, link 0, flags 800, type=1
> section(40) .debug_line, size 7374522, link 0, flags 800, type=1
> section(41) .debug_frame, size 702463, link 0, flags 800, type=1
> section(42) .debug_str, size 1017606, link 0, flags 830, type=1
> section(43) .debug_loc, size 3019453, link 0, flags 800, type=1
> section(44) .debug_ranges, size 1744583, link 0, flags 800, type=1
> section(45) .symtab, size 2955888, link 46, flags 0, type=2
> section(46) .strtab, size 2613072, link 0, flags 0, type=3
> section(47) .shstrtab, size 525, link 0, flags 0, type=3
> adding symbol seq_file
> adding symbol bpf_map
> adding symbol task_struct
> adding symbol file
> adding symbol bpf_prog
> adding symbol bpf_ctx_convert
> adding symbol sk_buff
> adding symbol xdp_buff
> adding symbol inet_sock
> adding symbol inet_connection_sock
> adding symbol inet_request_sock
> adding symbol inet_timewait_sock
> adding symbol request_sock
> adding symbol sock
> adding symbol sock_common
> adding symbol tcp_sock
> adding symbol tcp_request_sock
> adding symbol tcp_timewait_sock
> adding symbol tcp6_sock
> adding symbol udp_sock
> adding symbol udp6_sock
> adding symbol netlink_sock
> adding symbol fib6_info
> patching addr 36: ID 21502 [xdp_buff]
> patching addr 84: ID 63192 [udp_sock]
> patching addr 88: ID 63195 [udp6_sock]
> patching addr 76: ID 66968 [tcp_timewait_sock]
> patching addr 68: ID 61353 [tcp_sock]
> patching addr 72: ID 61567 [tcp_request_sock]
> patching addr 80: ID 63196 [tcp6_sock]
> patching addr 12: ID 169 [task_struct]
> patching addr 28: ID 169 [task_struct]
> patching addr 64: ID 4401 [sock_common]
> patching addr 60: ID 2894 [sock]
> patching addr 32: ID 3116 [sk_buff]
> patching addr 0: ID 1683 [seq_file]
> patching addr 4: ID 1683 [seq_file]
> patching addr 56: ID 4458 [request_sock]
> patching addr 92: ID 65748 [netlink_sock]
> patching addr 52: ID 66629 [inet_timewait_sock]
> patching addr 40: ID 37652 [inet_sock]
> patching addr 48: ID 61566 [inet_request_sock]
> patching addr 44: ID 61337 [inet_connection_sock]
> patching addr 16: ID 491 [file]
> patching addr 96: ID 56653 [fib6_info]
> patching addr 20: ID 3099 [bpf_prog]
> patching addr 8: ID 1926 [bpf_map]
> patching addr 24: ID 21629 [bpf_ctx_convert]
> FAILED elf_update(WRITE): invalid section alignment
> update failed for vmlinux.err.bak
>
>
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index e6e2d9e5ff48..718b2c0ee7ea 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -227,6 +227,7 @@ cleanup()
> rm -f .tmp_System.map
> rm -f .tmp_vmlinux*
> rm -f System.map
> + cp vmlinux vmlinux.err.bak
> rm -f vmlinux
> rm -f vmlinux.o
> }
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-18 9:14 ` Jiri Olsa
@ 2020-08-18 10:56 ` Jiri Olsa
2020-08-18 13:45 ` Jiri Olsa
0 siblings, 1 reply; 10+ messages in thread
From: Jiri Olsa @ 2020-08-18 10:56 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: linux-kernel, netdev, sdf, andriin
On Tue, Aug 18, 2020 at 11:14:10AM +0200, Jiri Olsa wrote:
> On Tue, Aug 18, 2020 at 10:55:55AM +0200, Jesper Dangaard Brouer wrote:
> >
> > On latest DaveM net-git tree (06a4ec1d9dc652), after linking (LD vmlinux) the
> > "BTFIDS vmlinux" fails. Are anybody else experiencing this? Are there already a
> > fix? (just returned from vacation so not fully up-to-date on ML yet)
> >
> > The tool which is called and error message:
> > ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > FAILED elf_update(WRITE): invalid section alignment
>
> hi,
> could you send your .config as well?
reproduced.. checking on fix
thanks,
jirka
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-18 10:56 ` Jiri Olsa
@ 2020-08-18 13:45 ` Jiri Olsa
2020-08-18 16:33 ` Jesper Dangaard Brouer
0 siblings, 1 reply; 10+ messages in thread
From: Jiri Olsa @ 2020-08-18 13:45 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: linux-kernel, netdev, sdf, andriin, Mark Wielaard
On Tue, Aug 18, 2020 at 12:56:08PM +0200, Jiri Olsa wrote:
> On Tue, Aug 18, 2020 at 11:14:10AM +0200, Jiri Olsa wrote:
> > On Tue, Aug 18, 2020 at 10:55:55AM +0200, Jesper Dangaard Brouer wrote:
> > >
> > > On latest DaveM net-git tree (06a4ec1d9dc652), after linking (LD vmlinux) the
> > > "BTFIDS vmlinux" fails. Are anybody else experiencing this? Are there already a
> > > fix? (just returned from vacation so not fully up-to-date on ML yet)
> > >
> > > The tool which is called and error message:
> > > ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > FAILED elf_update(WRITE): invalid section alignment
> >
> > hi,
> > could you send your .config as well?
>
> reproduced.. checking on fix
I discussed this with Mark (cc-ed) it seems to be a problem
with linker when dealing with compressed debug info data,
which is enabled in your .config
it works for me when I disable CONFIG_DEBUG_INFO_COMPRESSED option
Mark will fix this upstream, meanwhile he suggested workaround
we can do in resolve_btfids tool, that I'll try to send shortly
thanks,
jirka
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-18 13:45 ` Jiri Olsa
@ 2020-08-18 16:33 ` Jesper Dangaard Brouer
2020-08-18 17:29 ` Mark Wielaard
0 siblings, 1 reply; 10+ messages in thread
From: Jesper Dangaard Brouer @ 2020-08-18 16:33 UTC (permalink / raw)
To: Jiri Olsa; +Cc: brouer, linux-kernel, netdev, sdf, andriin, Mark Wielaard
On Tue, 18 Aug 2020 15:45:43 +0200
Jiri Olsa <jolsa@redhat.com> wrote:
> On Tue, Aug 18, 2020 at 12:56:08PM +0200, Jiri Olsa wrote:
> > On Tue, Aug 18, 2020 at 11:14:10AM +0200, Jiri Olsa wrote:
> > > On Tue, Aug 18, 2020 at 10:55:55AM +0200, Jesper Dangaard Brouer wrote:
> > > >
> > > > On latest DaveM net-git tree (06a4ec1d9dc652), after linking (LD vmlinux) the
> > > > "BTFIDS vmlinux" fails. Are anybody else experiencing this? Are there already a
> > > > fix? (just returned from vacation so not fully up-to-date on ML yet)
> > > >
> > > > The tool which is called and error message:
> > > > ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED elf_update(WRITE): invalid section alignment
> > >
> > > hi,
> > > could you send your .config as well?
> >
> > reproduced.. checking on fix
>
> I discussed this with Mark (cc-ed) it seems to be a problem
> with linker when dealing with compressed debug info data,
> which is enabled in your .config
>
> it works for me when I disable CONFIG_DEBUG_INFO_COMPRESSED option
Thanks for finding this!
I confirm that disabling CONFIG_DEBUG_INFO_COMPRESSED fixed the issue.
> Mark will fix this upstream, meanwhile he suggested workaround
> we can do in resolve_btfids tool, that I'll try to send shortly
Great!
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-18 16:33 ` Jesper Dangaard Brouer
@ 2020-08-18 17:29 ` Mark Wielaard
2020-08-19 15:34 ` Nick Clifton
0 siblings, 1 reply; 10+ messages in thread
From: Mark Wielaard @ 2020-08-18 17:29 UTC (permalink / raw)
To: Jesper Dangaard Brouer, Jiri Olsa
Cc: linux-kernel, netdev, sdf, andriin, nickc
Hi,
Adding Nick, the binutils maintainer, so we can make sure
binutils/elfutils agree on some ELF section compression corner case.
On Tue, 2020-08-18 at 18:33 +0200, Jesper Dangaard Brouer wrote:
> On Tue, 18 Aug 2020 15:45:43 +0200
> Jiri Olsa <jolsa@redhat.com> wrote:
>
> > On Tue, Aug 18, 2020 at 12:56:08PM +0200, Jiri Olsa wrote:
> > > On Tue, Aug 18, 2020 at 11:14:10AM +0200, Jiri Olsa wrote:
> > > > On Tue, Aug 18, 2020 at 10:55:55AM +0200, Jesper Dangaard Brouer wrote:
> > > > >
> > > > > On latest DaveM net-git tree (06a4ec1d9dc652), after linking (LD vmlinux) the
> > > > > "BTFIDS vmlinux" fails. Are anybody else experiencing this? Are there already a
> > > > > fix? (just returned from vacation so not fully up-to-date on ML yet)
> > > > >
> > > > > The tool which is called and error message:
> > > > > ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > > FAILED elf_update(WRITE): invalid section alignment
> > > >
> > > > hi,
> > > > could you send your .config as well?
> > >
> > > reproduced.. checking on fix
> >
> > I discussed this with Mark (cc-ed) it seems to be a problem
> > with linker when dealing with compressed debug info data,
> > which is enabled in your .config
> >
> > it works for me when I disable CONFIG_DEBUG_INFO_COMPRESSED option
>
> Thanks for finding this!
> I confirm that disabling CONFIG_DEBUG_INFO_COMPRESSED fixed the issue.
>
> > Mark will fix this upstream, meanwhile he suggested workaround
> > we can do in resolve_btfids tool, that I'll try to send shortly
>
> Great!
So, the issue is that there is some confusion about the correct
alignment of compressed ELF sections.
When an ELF section is compressed using gabi-zlib it contains a header
(a Elf_Chdr32 or Elf_Chdr64, followed by the compressed data) The
header explains how the section data is compressed, what the exploded
size is, what the alignment of that uncompressed data is, etc.
Because of this header the section data should be aligned to 4 (for
32bit) or 8 (for 64 bit) bytes, but binutils ld sets sh_addralign to 1.
[*]
elfutils libelf is liberal in what it accepts, and internally fixes up
the alignment if it is wrong. Which is why we probably didn't see this
before. But it won't let you write out misaligned data like that. Which
is slightly confusing, because if you didn't change that section data,
it is not immediately clear why you are getting an error.
Also if you would decompress the section data to use it and then
recompress it elfutils libelf would set sh_addralign correctly for you.
But it won't if you don't use the (uncompressed) data.
The workaround would be to explicitly set the alignment of the
compressed section before writing out the section. Which is what Jiri
is now testing.
But it would obviously be better if that wasn't necessary. So I'll try
to fix libelf so that if it fixes up the alignment when reading the
compressed data, it also does that when writing out the data again. But
that would only help for a new version of elfutils.
So it would be nice if binutils ld could also be fixed to write out
compressed sections with the correct alignment.
Then hopefully if someone has either a new elfutils or a new binutils
it just works without needing any workarounds.
Cheers,
Mark
[*] If this sounds vaguely familiar then that is because we did have a
different alignment bug, but for the uncompressed data (which is the
alignment set in the compression header):
https://bugzilla.redhat.com/show_bug.cgi?id=1678204
That bug was about ch_addralign, this bug is about sh_addralign.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-18 17:29 ` Mark Wielaard
@ 2020-08-19 15:34 ` Nick Clifton
2020-08-19 17:18 ` Jiri Olsa
0 siblings, 1 reply; 10+ messages in thread
From: Nick Clifton @ 2020-08-19 15:34 UTC (permalink / raw)
To: Mark Wielaard, Jesper Dangaard Brouer, Jiri Olsa
Cc: linux-kernel, netdev, sdf, andriin
Hi Mark,
> Adding Nick, the binutils maintainer, so we can make sure
> binutils/elfutils agree on some ELF section compression corner case.
Thanks for doing this.
> But it would obviously be better if that wasn't necessary. So I'll try
> to fix libelf so that if it fixes up the alignment when reading the
> compressed data, it also does that when writing out the data again. But
> that would only help for a new version of elfutils.
>
> So it would be nice if binutils ld could also be fixed to write out
> compressed sections with the correct alignment.
OK, I will look into doing this.
By any chance is there a small test case that you are using to check
this behaviour ? If so, please may I have a copy for myself ?
Cheers
Nick
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-19 15:34 ` Nick Clifton
@ 2020-08-19 17:18 ` Jiri Olsa
2020-08-19 22:00 ` Mark Wielaard
0 siblings, 1 reply; 10+ messages in thread
From: Jiri Olsa @ 2020-08-19 17:18 UTC (permalink / raw)
To: Nick Clifton
Cc: Mark Wielaard, Jesper Dangaard Brouer, linux-kernel, netdev, sdf,
andriin
On Wed, Aug 19, 2020 at 04:34:30PM +0100, Nick Clifton wrote:
> Hi Mark,
>
> > Adding Nick, the binutils maintainer, so we can make sure
> > binutils/elfutils agree on some ELF section compression corner case.
>
> Thanks for doing this.
>
> > But it would obviously be better if that wasn't necessary. So I'll try
> > to fix libelf so that if it fixes up the alignment when reading the
> > compressed data, it also does that when writing out the data again. But
> > that would only help for a new version of elfutils.
> >
> > So it would be nice if binutils ld could also be fixed to write out
> > compressed sections with the correct alignment.
>
> OK, I will look into doing this.
>
> By any chance is there a small test case that you are using to check
> this behaviour ? If so, please may I have a copy for myself ?
so when I take empty object and compile like:
$ echo 'int main(int argc, char **argv) { return 0; }' | gcc -c -o ex.o -g -gz=zlib -x c -
$ ld -o ex --compress-debug-sections=zlib ex.o
then there's .debug_info section that shows sh_addralign = 1
after I open the 'ex' obejct with elf_begin and iterate sections
according to Mark that should be 8 (on 64 bits)
when I change it to 8, the elf_update call won't fail for me
on that elf file
thanks,
jirka
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-19 17:18 ` Jiri Olsa
@ 2020-08-19 22:00 ` Mark Wielaard
2020-08-20 12:14 ` Nick Clifton
0 siblings, 1 reply; 10+ messages in thread
From: Mark Wielaard @ 2020-08-19 22:00 UTC (permalink / raw)
To: Jiri Olsa, Nick Clifton
Cc: Jesper Dangaard Brouer, linux-kernel, netdev, sdf, andriin
[-- Attachment #1: Type: text/plain, Size: 3992 bytes --]
Hi,
On Wed, 2020-08-19 at 19:18 +0200, Jiri Olsa wrote:
> On Wed, Aug 19, 2020 at 04:34:30PM +0100, Nick Clifton wrote:
> > > So it would be nice if binutils ld could also be fixed to write out
> > > compressed sections with the correct alignment.
> >
> > OK, I will look into doing this.
> >
> > By any chance is there a small test case that you are using to check
> > this behaviour ? If so, please may I have a copy for myself ?
>
> so when I take empty object and compile like:
>
> $ echo 'int main(int argc, char **argv) { return 0; }' | gcc -c -o ex.o -g -gz=zlib -x c -
> $ ld -o ex --compress-debug-sections=zlib ex.o
>
> then there's .debug_info section that shows sh_addralign = 1
Specifically, if you extend the example code a bit so that it has a
couple more interesting compressed .debug sections (like in an vmlinux
image) you'll see, eu-readelf -Sz:
Section Headers:
[Nr] Name Type Addr Off Size ES Flags Lk Inf Al
[Compression Size Al]
[37] .debug_aranges PROGBITS 0000000000000000 027ae9f0 0000b274 0 C 0 0 16
[ELF ZLIB (1) 00028030 16]
[38] .debug_info PROGBITS 0000000000000000 027b9c64 07b1fc3d 0 C 0 0 1
[ELF ZLIB (1) 0cb137ad 1]
[39] .debug_abbrev PROGBITS 0000000000000000 0a2d98a1 00119647 0 C 0 0 1
[ELF ZLIB (1) 0060811f 1]
[40] .debug_line PROGBITS 0000000000000000 0a3f2ee8 007086ba 0 C 0 0 1
[ELF ZLIB (1) 01557659 1]
[41] .debug_frame PROGBITS 0000000000000000 0aafb5a8 000ab7ff 0 C 0 0 8
[ELF ZLIB (1) 002a6bf8 8]
[42] .debug_str PROGBITS 0000000000000000 0aba6da7 000f86e3 1 MSC 0 0 1
[ELF ZLIB (1) 003a8a8e 1]
[43] .debug_loc PROGBITS 0000000000000000 0ac9f48a 002e12bd 0 C 0 0 1
[ELF ZLIB (1) 00e0c448 1]
[44] .debug_ranges PROGBITS 0000000000000000 0af80750 001a9ec7 0 C 0 0 16
[ELF ZLIB (1) 00e84b20 16]
Note that the sh_addralign of the sections is set to the same valie as
ch_addralign. That is the alignment of the decompressed data, what
sh_addralign would have been if it wasn't a compressed section.
The sh_addralign of a compressed section however should be equal to
alignment for the datastructure inside it, either 4, for 32bit:
typedef struct
{
Elf32_Word ch_type; /* Compression format. */
Elf32_Word ch_size; /* Uncompressed data size. */
Elf32_Word ch_addralign; /* Uncompressed data alignment. */
} Elf32_Chdr;
or 8, for 64bit:
typedef struct
{
Elf64_Word ch_type; /* Compression format. */
Elf64_Word ch_reserved;
Elf64_Xword ch_size; /* Uncompressed data size. */
Elf64_Xword ch_addralign; /* Uncompressed data alignment. */
} Elf64_Chdr;
At least, that is what elfutils libelf expects. And which I believe is
what the ELF spec implies when it says:
The sh_size and sh_addralign fields of the section header for a
compressed section reflect the requirements of the compressed
section. The ch_size and ch_addralign fields in the compression
header provide the corresponding values for the uncompressed data,
thereby supplying the values that sh_size and sh_addralign would
have had if the section had not been compressed.
> after I open the 'ex' obejct with elf_begin and iterate sections
>
> according to Mark that should be 8 (on 64 bits)
>
> when I change it to 8, the elf_update call won't fail for me
> on that elf file
Right, I have a patch that fixes it up in libelf, see attached.
That should make things work without needing a workaround. But of
course I just posted it and it isn't even upstream yet. So for now the
workaround will be needed and it would be nice if binutils ld could
also be fixed to set the sh_addralign field correctly.
Cheers,
Mark
[-- Attachment #2: 0001-libelf-Fixup-SHF_COMPRESSED-sh_addralign-in-elf_upda.patch --]
[-- Type: text/x-patch, Size: 2712 bytes --]
From 55c5c9a568ed707bcea1388bf3a525212d8cf4b8 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Wed, 19 Aug 2020 23:41:24 +0200
Subject: [PATCH] libelf: Fixup SHF_COMPRESSED sh_addralign in elf_update if
necessary.
In elf_getdata.c we have the following to compensate for possibly
bad sh_addralign values of compressed sections:
/* Compressed data has a header, but then compressed data.
Make sure to set the alignment of the header explicitly,
don't trust the file alignment for the section, it is
often wrong. */
if ((flags & SHF_COMPRESSED) != 0)
{
entsize = 1;
align = __libelf_type_align (elf->class, ELF_T_CHDR);
}
Which makes sure the d_data alignment is correct for the Chdr struct
at the start of the compressed section.
But this means that if a user just reads such a compressed section
without changing it, and then tries to write it out again using
elf_update they get an error message about d_align and sh_addralign
being out of sync.
We already correct obviously incorrect sh_entsize fields.
Do the same for the sh_addralign field of a SHF_COMPRESSED section.
Signed-off-by: Mark Wielaard <mark@klomp.org>
---
libelf/ChangeLog | 5 +++++
libelf/elf32_updatenull.c | 12 ++++++++++++
2 files changed, 17 insertions(+)
diff --git a/libelf/ChangeLog b/libelf/ChangeLog
index 8f6d2d2d..77044c1c 100644
--- a/libelf/ChangeLog
+++ b/libelf/ChangeLog
@@ -1,3 +1,8 @@
+2020-08-19 Mark Wielaard <mark@klomp.org>
+
+ * elf32_updatenull.c (updatenull_wrlock): Fixup the sh_addralign
+ of an SHF_COMPRESSED section if necessary.
+
2020-06-04 Mark Wielaard <mark@klomp.org>
* elf.h: Update from glibc.
diff --git a/libelf/elf32_updatenull.c b/libelf/elf32_updatenull.c
index 5f3cdbf6..d0d4d1eb 100644
--- a/libelf/elf32_updatenull.c
+++ b/libelf/elf32_updatenull.c
@@ -267,6 +267,18 @@ __elfw2(LIBELFBITS,updatenull_wrlock) (Elf *elf, int *change_bop, size_t shnum)
update_if_changed (shdr->sh_entsize, sh_entsize,
scn->shdr_flags);
+ /* Likewise for the alignment of a compressed section.
+ For a SHF_COMPRESSED section set the correct
+ sh_addralign value, which must match the d_align of
+ the data (see __libelf_set_rawdata in elf_getdata.c). */
+ if ((shdr->sh_flags & SHF_COMPRESSED) != 0)
+ {
+ sh_align = __libelf_type_align (ELFW(ELFCLASS,LIBELFBITS),
+ ELF_T_CHDR);
+ update_if_changed (shdr->sh_addralign, sh_align,
+ scn->shdr_flags);
+ }
+
if (scn->data_read == 0
&& __libelf_set_rawdata_wrlock (scn) != 0)
/* Something went wrong. The error value is already set. */
--
2.18.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: Kernel build error on BTFIDS vmlinux
2020-08-19 22:00 ` Mark Wielaard
@ 2020-08-20 12:14 ` Nick Clifton
0 siblings, 0 replies; 10+ messages in thread
From: Nick Clifton @ 2020-08-20 12:14 UTC (permalink / raw)
To: Mark Wielaard, Jiri Olsa
Cc: Jesper Dangaard Brouer, linux-kernel, netdev, sdf, andriin
Hi Guys,
>> so when I take empty object and compile like:
>>
>> $ echo 'int main(int argc, char **argv) { return 0; }' | gcc -c -o ex.o -g -gz=zlib -x c -
>> $ ld -o ex --compress-debug-sections=zlib ex.o
Thanks Mark. I have now created a binutils PR for this bug, and I am looking into a fix:
https://sourceware.org/bugzilla/show_bug.cgi?id=26428
Cheers
Nick
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-08-20 12:15 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-18 8:55 Kernel build error on BTFIDS vmlinux Jesper Dangaard Brouer
2020-08-18 9:14 ` Jiri Olsa
2020-08-18 10:56 ` Jiri Olsa
2020-08-18 13:45 ` Jiri Olsa
2020-08-18 16:33 ` Jesper Dangaard Brouer
2020-08-18 17:29 ` Mark Wielaard
2020-08-19 15:34 ` Nick Clifton
2020-08-19 17:18 ` Jiri Olsa
2020-08-19 22:00 ` Mark Wielaard
2020-08-20 12:14 ` Nick Clifton
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).