* [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF
@ 2020-03-16 22:25 Stanislav Fomichev
2020-03-16 23:15 ` Fangrui Song
2020-03-16 23:57 ` Nathan Chancellor
0 siblings, 2 replies; 7+ messages in thread
From: Stanislav Fomichev @ 2020-03-16 22:25 UTC (permalink / raw)
To: netdev, bpf
Cc: davem, ast, daniel, Stanislav Fomichev, Nick Desaulniers,
Nathan Chancellor
Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux
BTF") switched from --dump-section to
--only-section/--change-section-address for BTF export assuming
those ("legacy") options should cover all objcopy versions.
Turns out llvm-objcopy doesn't implement --change-section-address [1],
but it does support --dump-section. Let's partially roll back and
try to use --dump-section first and fall back to
--only-section/--change-section-address for the older binutils.
1. https://bugs.llvm.org/show_bug.cgi?id=45217
Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux")
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Reported-by: Nathan Chancellor <natechancellor@gmail.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/871
Signed-off-by: Stanislav Fomichev <sdf@google.com>
---
scripts/link-vmlinux.sh | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index dd484e92752e..8ddf57cbc439 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -127,6 +127,16 @@ gen_btf()
cut -d, -f1 | cut -d' ' -f2)
bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \
awk '{print $4}')
+
+ # Compatibility issues:
+ # - pre-2.25 binutils objcopy doesn't support --dump-section
+ # - llvm-objcopy doesn't support --change-section-address, but
+ # does support --dump-section
+ #
+ # Try to use --dump-section which should cover both recent
+ # binutils and llvm-objcopy and fall back to --only-section
+ # for pre-2.25 binutils.
+ ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \
${OBJCOPY} --change-section-address .BTF=0 \
--set-section-flags .BTF=alloc -O binary \
--only-section=.BTF ${1} .btf.vmlinux.bin
--
2.25.1.481.gfbce0eb801-goog
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF 2020-03-16 22:25 [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF Stanislav Fomichev @ 2020-03-16 23:15 ` Fangrui Song 2020-03-16 23:56 ` Stanislav Fomichev 2020-03-16 23:57 ` Nathan Chancellor 1 sibling, 1 reply; 7+ messages in thread From: Fangrui Song @ 2020-03-16 23:15 UTC (permalink / raw) To: Stanislav Fomichev Cc: netdev, bpf, davem, ast, daniel, Nick Desaulniers, Nathan Chancellor, clang-built-linux On 2020-03-16, Stanislav Fomichev wrote: >Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux >BTF") switched from --dump-section to >--only-section/--change-section-address for BTF export assuming >those ("legacy") options should cover all objcopy versions. > >Turns out llvm-objcopy doesn't implement --change-section-address [1], >but it does support --dump-section. Let's partially roll back and >try to use --dump-section first and fall back to >--only-section/--change-section-address for the older binutils. > >1. https://bugs.llvm.org/show_bug.cgi?id=45217 > >Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux") >Tested-by: Nick Desaulniers <ndesaulniers@google.com> >Reported-by: Nathan Chancellor <natechancellor@gmail.com> >Link: https://github.com/ClangBuiltLinux/linux/issues/871 >Signed-off-by: Stanislav Fomichev <sdf@google.com> >--- > scripts/link-vmlinux.sh | 10 ++++++++++ > 1 file changed, 10 insertions(+) > >diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh >index dd484e92752e..8ddf57cbc439 100755 >--- a/scripts/link-vmlinux.sh >+++ b/scripts/link-vmlinux.sh >@@ -127,6 +127,16 @@ gen_btf() > cut -d, -f1 | cut -d' ' -f2) > bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \ > awk '{print $4}') >+ >+ # Compatibility issues: >+ # - pre-2.25 binutils objcopy doesn't support --dump-section >+ # - llvm-objcopy doesn't support --change-section-address, but >+ # does support --dump-section >+ # >+ # Try to use --dump-section which should cover both recent >+ # binutils and llvm-objcopy and fall back to --only-section >+ # for pre-2.25 binutils. >+ ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \ > ${OBJCOPY} --change-section-address .BTF=0 \ > --set-section-flags .BTF=alloc -O binary \ > --only-section=.BTF ${1} .btf.vmlinux.bin >-- >2.25.1.481.gfbce0eb801-goog So let me take advantage of this email to ask some questions about commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux BTF"). Does .BTF have the SHF_ALLOC flag? Is it a GNU objcopy<2.25 bug that objcopy --set-section-flags .BTF=alloc -O binary --only-section=.BTF does not skip the content? Non-SHF_ALLOC sections usually have 0 sh_addr. Why do they need --change-section-address .BTF=0 at all? Regarding >Turns out llvm-objcopy doesn't implement --change-section-address [1], This option will be difficult to implement in llvm-objcopy if we intend it to have a GNU objcopy compatible behavior. Without --only-section, it is not very clear how --change-section-{address,vma,lma} will affect program headers. There will be a debate even if we decide to implement them in llvm-objcopy. Some PT_LOAD rewriting examples: objcopy --change-section-address .plt=0 a b objcopy --change-section-address .text=0 a b There is another bug related to -B (https://github.com/ClangBuiltLinux/linux/issues/871#issuecomment-599790909): + objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin + objcopy -I binary -O elf64-x86-64 -B x86_64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin.o objcopy: architecture x86_64 unknown + echo 'Failed to generate BTF for vmlinux' It should be i386:x86_64. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF 2020-03-16 23:15 ` Fangrui Song @ 2020-03-16 23:56 ` Stanislav Fomichev 2020-03-17 0:56 ` Fangrui Song 0 siblings, 1 reply; 7+ messages in thread From: Stanislav Fomichev @ 2020-03-16 23:56 UTC (permalink / raw) To: Fangrui Song Cc: Stanislav Fomichev, netdev, bpf, davem, ast, daniel, Nick Desaulniers, Nathan Chancellor, clang-built-linux On 03/16, Fangrui Song wrote: > On 2020-03-16, Stanislav Fomichev wrote: > > Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux > > BTF") switched from --dump-section to > > --only-section/--change-section-address for BTF export assuming > > those ("legacy") options should cover all objcopy versions. > > > > Turns out llvm-objcopy doesn't implement --change-section-address [1], > > but it does support --dump-section. Let's partially roll back and > > try to use --dump-section first and fall back to > > --only-section/--change-section-address for the older binutils. > > > > 1. https://bugs.llvm.org/show_bug.cgi?id=45217 > > > > Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux") > > Tested-by: Nick Desaulniers <ndesaulniers@google.com> > > Reported-by: Nathan Chancellor <natechancellor@gmail.com> > > Link: https://github.com/ClangBuiltLinux/linux/issues/871 > > Signed-off-by: Stanislav Fomichev <sdf@google.com> > > --- > > scripts/link-vmlinux.sh | 10 ++++++++++ > > 1 file changed, 10 insertions(+) > > > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh > > index dd484e92752e..8ddf57cbc439 100755 > > --- a/scripts/link-vmlinux.sh > > +++ b/scripts/link-vmlinux.sh > > @@ -127,6 +127,16 @@ gen_btf() > > cut -d, -f1 | cut -d' ' -f2) > > bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \ > > awk '{print $4}') > > + > > + # Compatibility issues: > > + # - pre-2.25 binutils objcopy doesn't support --dump-section > > + # - llvm-objcopy doesn't support --change-section-address, but > > + # does support --dump-section > > + # > > + # Try to use --dump-section which should cover both recent > > + # binutils and llvm-objcopy and fall back to --only-section > > + # for pre-2.25 binutils. > > + ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \ > > ${OBJCOPY} --change-section-address .BTF=0 \ > > --set-section-flags .BTF=alloc -O binary \ > > --only-section=.BTF ${1} .btf.vmlinux.bin > > -- > > 2.25.1.481.gfbce0eb801-goog > > So let me take advantage of this email to ask some questions about > commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux BTF"). > > Does .BTF have the SHF_ALLOC flag? No, that's why we manually do '--set-section-flags .BTF=alloc' to make --only-section work. > Is it a GNU objcopy<2.25 bug that objcopy --set-section-flags .BTF=alloc -O binary --only-section=.BTF does not skip the content? > Non-SHF_ALLOC sections usually have 0 sh_addr. Why do they need --change-section-address .BTF=0 at all? I think that '--set-section-flags .BTF=alloc' causes objcopy to put some non-zero (valid) sh_addr, that's why we need to reset it to 0. (it's not clear if it's a feature or a bug and man isn't helpful) > Regarding > > > Turns out llvm-objcopy doesn't implement --change-section-address [1], > > This option will be difficult to implement in llvm-objcopy if we intend > it to have a GNU objcopy compatible behavior. > Without --only-section, it is not very clear how > --change-section-{address,vma,lma} will affect program headers. > There will be a debate even if we decide to implement them in llvm-objcopy. > > Some PT_LOAD rewriting examples: > > objcopy --change-section-address .plt=0 a b > objcopy --change-section-address .text=0 a b > > There is another bug related to -B > (https://github.com/ClangBuiltLinux/linux/issues/871#issuecomment-599790909): > > + objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc > -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin > + objcopy -I binary -O elf64-x86-64 -B x86_64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin.o > objcopy: architecture x86_64 unknown > + echo 'Failed to generate BTF for vmlinux' > > It should be i386:x86_64. Here is what I get: + bin_arch=i386:x86-64 + bin_format=elf64-x86-64 + objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin + objcopy -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin. Can you try to see where your x86_64 is coming from? ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF 2020-03-16 23:56 ` Stanislav Fomichev @ 2020-03-17 0:56 ` Fangrui Song 2020-03-17 3:14 ` Andrii Nakryiko 0 siblings, 1 reply; 7+ messages in thread From: Fangrui Song @ 2020-03-17 0:56 UTC (permalink / raw) To: Stanislav Fomichev Cc: Stanislav Fomichev, netdev, bpf, davem, ast, daniel, Nick Desaulniers, Nathan Chancellor, clang-built-linux On 2020-03-16, Stanislav Fomichev wrote: >On 03/16, Fangrui Song wrote: >> On 2020-03-16, Stanislav Fomichev wrote: >> > Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux >> > BTF") switched from --dump-section to >> > --only-section/--change-section-address for BTF export assuming >> > those ("legacy") options should cover all objcopy versions. >> > >> > Turns out llvm-objcopy doesn't implement --change-section-address [1], >> > but it does support --dump-section. Let's partially roll back and >> > try to use --dump-section first and fall back to >> > --only-section/--change-section-address for the older binutils. >> > >> > 1. https://bugs.llvm.org/show_bug.cgi?id=45217 >> > >> > Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux") >> > Tested-by: Nick Desaulniers <ndesaulniers@google.com> >> > Reported-by: Nathan Chancellor <natechancellor@gmail.com> >> > Link: https://github.com/ClangBuiltLinux/linux/issues/871 >> > Signed-off-by: Stanislav Fomichev <sdf@google.com> >> > --- >> > scripts/link-vmlinux.sh | 10 ++++++++++ >> > 1 file changed, 10 insertions(+) >> > >> > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh >> > index dd484e92752e..8ddf57cbc439 100755 >> > --- a/scripts/link-vmlinux.sh >> > +++ b/scripts/link-vmlinux.sh >> > @@ -127,6 +127,16 @@ gen_btf() >> > cut -d, -f1 | cut -d' ' -f2) >> > bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \ >> > awk '{print $4}') >> > + >> > + # Compatibility issues: >> > + # - pre-2.25 binutils objcopy doesn't support --dump-section >> > + # - llvm-objcopy doesn't support --change-section-address, but >> > + # does support --dump-section >> > + # >> > + # Try to use --dump-section which should cover both recent >> > + # binutils and llvm-objcopy and fall back to --only-section >> > + # for pre-2.25 binutils. >> > + ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \ >> > ${OBJCOPY} --change-section-address .BTF=0 \ >> > --set-section-flags .BTF=alloc -O binary \ >> > --only-section=.BTF ${1} .btf.vmlinux.bin >> > -- >> > 2.25.1.481.gfbce0eb801-goog >> >> So let me take advantage of this email to ask some questions about >> commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux BTF"). >> >> Does .BTF have the SHF_ALLOC flag? >No, that's why we manually do '--set-section-flags .BTF=alloc' to >make --only-section work. > >> Is it a GNU objcopy<2.25 bug that objcopy --set-section-flags .BTF=alloc -O binary --only-section=.BTF does not skip the content? >> Non-SHF_ALLOC sections usually have 0 sh_addr. Why do they need --change-section-address .BTF=0 at all? >I think that '--set-section-flags .BTF=alloc' causes objcopy to put >some non-zero (valid) sh_addr, that's why we need to reset it to 0. > >(it's not clear if it's a feature or a bug and man isn't helpful) > >> Regarding >> >> > Turns out llvm-objcopy doesn't implement --change-section-address [1], >> >> This option will be difficult to implement in llvm-objcopy if we intend >> it to have a GNU objcopy compatible behavior. >> Without --only-section, it is not very clear how >> --change-section-{address,vma,lma} will affect program headers. >> There will be a debate even if we decide to implement them in llvm-objcopy. >> >> Some PT_LOAD rewriting examples: >> >> objcopy --change-section-address .plt=0 a b >> objcopy --change-section-address .text=0 a b >> >> There is another bug related to -B >> (https://github.com/ClangBuiltLinux/linux/issues/871#issuecomment-599790909): >> >> + objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc >> -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin >> + objcopy -I binary -O elf64-x86-64 -B x86_64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin.o >> objcopy: architecture x86_64 unknown >> + echo 'Failed to generate BTF for vmlinux' >> >> It should be i386:x86_64. >Here is what I get: > >+ bin_arch=i386:x86-64 >+ bin_format=elf64-x86-64 >+ objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin >+ objcopy -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin. > >Can you try to see where your x86_64 is coming from? llvm-objdump -f does not print bfdarch (ARCH= in binutils-gdb/ld/emulparams/*.sh). % objdump -f .btf.vmlinux.bin.o .btf.vmlinux.bin.o: file format elf64-x86-64 architecture: i386:x86-64, flags 0x00000010: HAS_SYMS start address 0x0000000000000000 % llvm-objdump -f .btf.vmlinux.bin.o .btf.vmlinux.bin.o: file format elf64-x86-64 architecture: x86_64 start address: 0x0000000000000000 % objcopy -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section .data=.BTF .btf.vmlinux.bin meow.btf.vmlinux.bin.o # happy % objcopy -I binary -O elf64-x86-64 -B x86-64 --rename-section .data=.BTF .btf.vmlinux.bin meow.btf.vmlinux.bin.o objcopy: architecture x86-64 unknown As a non-x86 example, elf64-powerpcle / powerpc:common64: % powerpc64le-linux-gnu-objdump -f meow.btf.vmlinux.bin.o meow.btf.vmlinux.bin.o: file format elf64-powerpcle architecture: powerpc:common64, flags 0x00000010: Unfortunately, GNU objcopy<2.34 (before I complained about the redundant -B https://sourceware.org/bugzilla/show_bug.cgi?id=24968) could not infer -B from -O elf* . % objcopy -I binary -O elf64-x86-64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin.o #<2.34 % file .btf.vmlinux.bin.o .btf.vmlinux.bin.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped objcopy: architecture x86-64 unknown GNU ld and lld will error for e_machine==0. I will be a bit nervous to make llvm-objdump behave more BFD like. Adding i386:x86-64, powerpc:common64, etc does not look particularly clean. Fortunately, looking at the code, it seems that we only want to retain .BTF The following scheme may be simpler: objcopy --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin.o && printf '\1' | dd of=.btf.vmlinux.bin.o conv=notrunc bs=1 seek=16 The command after && is to change e_type from ET_EXEC to ET_REL. GNU ld has an extremely rare feature that allows ET_EXEC to be linked, but lld is more rigid and will reject such an input file. https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/HHIUPUXRIZ3KLTK4TPLG2V4PFP32HRBE/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF 2020-03-17 0:56 ` Fangrui Song @ 2020-03-17 3:14 ` Andrii Nakryiko 0 siblings, 0 replies; 7+ messages in thread From: Andrii Nakryiko @ 2020-03-17 3:14 UTC (permalink / raw) To: Fangrui Song Cc: Stanislav Fomichev, Stanislav Fomichev, Networking, bpf, David S. Miller, Alexei Starovoitov, Daniel Borkmann, Nick Desaulniers, Nathan Chancellor, clang-built-linux On Mon, Mar 16, 2020 at 6:09 PM Fangrui Song <maskray@google.com> wrote: > > On 2020-03-16, Stanislav Fomichev wrote: > >On 03/16, Fangrui Song wrote: > >> On 2020-03-16, Stanislav Fomichev wrote: > >> > Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux > >> > BTF") switched from --dump-section to > >> > --only-section/--change-section-address for BTF export assuming > >> > those ("legacy") options should cover all objcopy versions. > >> > > >> > Turns out llvm-objcopy doesn't implement --change-section-address [1], > >> > but it does support --dump-section. Let's partially roll back and > >> > try to use --dump-section first and fall back to > >> > --only-section/--change-section-address for the older binutils. > >> > > >> > 1. https://bugs.llvm.org/show_bug.cgi?id=45217 > >> > > >> > Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux") > >> > Tested-by: Nick Desaulniers <ndesaulniers@google.com> > >> > Reported-by: Nathan Chancellor <natechancellor@gmail.com> > >> > Link: https://github.com/ClangBuiltLinux/linux/issues/871 > >> > Signed-off-by: Stanislav Fomichev <sdf@google.com> > >> > --- > >> > scripts/link-vmlinux.sh | 10 ++++++++++ > >> > 1 file changed, 10 insertions(+) > >> > > >> > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh > >> > index dd484e92752e..8ddf57cbc439 100755 > >> > --- a/scripts/link-vmlinux.sh > >> > +++ b/scripts/link-vmlinux.sh > >> > @@ -127,6 +127,16 @@ gen_btf() > >> > cut -d, -f1 | cut -d' ' -f2) > >> > bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \ > >> > awk '{print $4}') > >> > + > >> > + # Compatibility issues: > >> > + # - pre-2.25 binutils objcopy doesn't support --dump-section > >> > + # - llvm-objcopy doesn't support --change-section-address, but > >> > + # does support --dump-section > >> > + # > >> > + # Try to use --dump-section which should cover both recent > >> > + # binutils and llvm-objcopy and fall back to --only-section > >> > + # for pre-2.25 binutils. > >> > + ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \ > >> > ${OBJCOPY} --change-section-address .BTF=0 \ > >> > --set-section-flags .BTF=alloc -O binary \ > >> > --only-section=.BTF ${1} .btf.vmlinux.bin > >> > -- > >> > 2.25.1.481.gfbce0eb801-goog > >> > >> So let me take advantage of this email to ask some questions about > >> commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux BTF"). > >> > >> Does .BTF have the SHF_ALLOC flag? > >No, that's why we manually do '--set-section-flags .BTF=alloc' to > >make --only-section work. > > > >> Is it a GNU objcopy<2.25 bug that objcopy --set-section-flags .BTF=alloc -O binary --only-section=.BTF does not skip the content? > >> Non-SHF_ALLOC sections usually have 0 sh_addr. Why do they need --change-section-address .BTF=0 at all? > >I think that '--set-section-flags .BTF=alloc' causes objcopy to put > >some non-zero (valid) sh_addr, that's why we need to reset it to 0. > > > >(it's not clear if it's a feature or a bug and man isn't helpful) > > > >> Regarding > >> > >> > Turns out llvm-objcopy doesn't implement --change-section-address [1], > >> > >> This option will be difficult to implement in llvm-objcopy if we intend > >> it to have a GNU objcopy compatible behavior. > >> Without --only-section, it is not very clear how > >> --change-section-{address,vma,lma} will affect program headers. > >> There will be a debate even if we decide to implement them in llvm-objcopy. > >> > >> Some PT_LOAD rewriting examples: > >> > >> objcopy --change-section-address .plt=0 a b > >> objcopy --change-section-address .text=0 a b > >> > >> There is another bug related to -B > >> (https://github.com/ClangBuiltLinux/linux/issues/871#issuecomment-599790909): > >> > >> + objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc > >> -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin > >> + objcopy -I binary -O elf64-x86-64 -B x86_64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin.o > >> objcopy: architecture x86_64 unknown > >> + echo 'Failed to generate BTF for vmlinux' > >> > >> It should be i386:x86_64. > >Here is what I get: > > > >+ bin_arch=i386:x86-64 > >+ bin_format=elf64-x86-64 > >+ objcopy --change-section-address .BTF=0 --set-section-flags .BTF=alloc -O binary --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin > >+ objcopy -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin. > > > >Can you try to see where your x86_64 is coming from? > > llvm-objdump -f does not print bfdarch (ARCH= in binutils-gdb/ld/emulparams/*.sh). > > % objdump -f .btf.vmlinux.bin.o > > .btf.vmlinux.bin.o: file format elf64-x86-64 > architecture: i386:x86-64, flags 0x00000010: > HAS_SYMS > start address 0x0000000000000000 > > % llvm-objdump -f .btf.vmlinux.bin.o > > .btf.vmlinux.bin.o: file format elf64-x86-64 > > architecture: x86_64 > start address: 0x0000000000000000 > > % objcopy -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section .data=.BTF .btf.vmlinux.bin meow.btf.vmlinux.bin.o > # happy > % objcopy -I binary -O elf64-x86-64 -B x86-64 --rename-section .data=.BTF .btf.vmlinux.bin meow.btf.vmlinux.bin.o > objcopy: architecture x86-64 unknown > > > As a non-x86 example, elf64-powerpcle / powerpc:common64: > > % powerpc64le-linux-gnu-objdump -f meow.btf.vmlinux.bin.o > > meow.btf.vmlinux.bin.o: file format elf64-powerpcle > architecture: powerpc:common64, flags 0x00000010: > > > Unfortunately, GNU objcopy<2.34 (before I complained about the redundant -B https://sourceware.org/bugzilla/show_bug.cgi?id=24968) > could not infer -B from -O elf* . > % objcopy -I binary -O elf64-x86-64 --rename-section .data=.BTF .btf.vmlinux.bin .btf.vmlinux.bin.o #<2.34 > % file .btf.vmlinux.bin.o > .btf.vmlinux.bin.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped > objcopy: architecture x86-64 unknown > > GNU ld and lld will error for e_machine==0. > > > > I will be a bit nervous to make llvm-objdump behave more BFD like. > Adding i386:x86-64, powerpc:common64, etc does not look particularly clean. > Fortunately, looking at the code, it seems that we only want to retain .BTF > The following scheme may be simpler: > > objcopy --only-section=.BTF .tmp_vmlinux.btf .btf.vmlinux.bin.o && printf '\1' | dd of=.btf.vmlinux.bin.o conv=notrunc bs=1 seek=16 > This part looks pretty crazy. Would it be simpler to detect whether $(OBJCOPY) is llvm-objcopy and handle that in a bit less hacky way? > The command after && is to change e_type from ET_EXEC to ET_REL. GNU ld has an extremely rare feature that allows ET_EXEC to be linked, > but lld is more rigid and will reject such an input file. > https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/HHIUPUXRIZ3KLTK4TPLG2V4PFP32HRBE/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF 2020-03-16 22:25 [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF Stanislav Fomichev 2020-03-16 23:15 ` Fangrui Song @ 2020-03-16 23:57 ` Nathan Chancellor 2020-03-17 0:01 ` Stanislav Fomichev 1 sibling, 1 reply; 7+ messages in thread From: Nathan Chancellor @ 2020-03-16 23:57 UTC (permalink / raw) To: Stanislav Fomichev; +Cc: netdev, bpf, davem, ast, daniel, Nick Desaulniers On Mon, Mar 16, 2020 at 03:25:18PM -0700, Stanislav Fomichev wrote: > Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux > BTF") switched from --dump-section to > --only-section/--change-section-address for BTF export assuming > those ("legacy") options should cover all objcopy versions. > > Turns out llvm-objcopy doesn't implement --change-section-address [1], > but it does support --dump-section. Let's partially roll back and > try to use --dump-section first and fall back to > --only-section/--change-section-address for the older binutils. > > 1. https://bugs.llvm.org/show_bug.cgi?id=45217 > > Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux") > Tested-by: Nick Desaulniers <ndesaulniers@google.com> > Reported-by: Nathan Chancellor <natechancellor@gmail.com> > Link: https://github.com/ClangBuiltLinux/linux/issues/871 > Signed-off-by: Stanislav Fomichev <sdf@google.com> > --- > scripts/link-vmlinux.sh | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh > index dd484e92752e..8ddf57cbc439 100755 > --- a/scripts/link-vmlinux.sh > +++ b/scripts/link-vmlinux.sh > @@ -127,6 +127,16 @@ gen_btf() > cut -d, -f1 | cut -d' ' -f2) > bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \ > awk '{print $4}') > + > + # Compatibility issues: > + # - pre-2.25 binutils objcopy doesn't support --dump-section > + # - llvm-objcopy doesn't support --change-section-address, but > + # does support --dump-section > + # > + # Try to use --dump-section which should cover both recent > + # binutils and llvm-objcopy and fall back to --only-section > + # for pre-2.25 binutils. > + ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \ > ${OBJCOPY} --change-section-address .BTF=0 \ > --set-section-flags .BTF=alloc -O binary \ > --only-section=.BTF ${1} .btf.vmlinux.bin > -- > 2.25.1.481.gfbce0eb801-goog > Hi Stanislav, Thank you for the patch! This is targeting the bpf tree but it uses the bin_file variable that comes from commit af73d78bd384 ("kbuild: Remove debug info from kallsyms linking") in the bpf-next tree. In this form, when applied to mainline, the first command fails because $bin_file doesn't exist and falls back to the second command, which results in no net change for llvm-objcopy. When manually applied to next-20200316 or applied to mainline with $bin_file replaced with .btf.vmlinux.bin, x86_64 and aarch64 successfully generate BTF with tip of tree llvm-objcopy. Tested-by: Nathan Chancellor <natechancellor@gmail.com> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF 2020-03-16 23:57 ` Nathan Chancellor @ 2020-03-17 0:01 ` Stanislav Fomichev 0 siblings, 0 replies; 7+ messages in thread From: Stanislav Fomichev @ 2020-03-17 0:01 UTC (permalink / raw) To: Nathan Chancellor Cc: Stanislav Fomichev, netdev, bpf, davem, ast, daniel, Nick Desaulniers On 03/16, Nathan Chancellor wrote: > On Mon, Mar 16, 2020 at 03:25:18PM -0700, Stanislav Fomichev wrote: > > Commit da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux > > BTF") switched from --dump-section to > > --only-section/--change-section-address for BTF export assuming > > those ("legacy") options should cover all objcopy versions. > > > > Turns out llvm-objcopy doesn't implement --change-section-address [1], > > but it does support --dump-section. Let's partially roll back and > > try to use --dump-section first and fall back to > > --only-section/--change-section-address for the older binutils. > > > > 1. https://bugs.llvm.org/show_bug.cgi?id=45217 > > > > Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux") > > Tested-by: Nick Desaulniers <ndesaulniers@google.com> > > Reported-by: Nathan Chancellor <natechancellor@gmail.com> > > Link: https://github.com/ClangBuiltLinux/linux/issues/871 > > Signed-off-by: Stanislav Fomichev <sdf@google.com> > > --- > > scripts/link-vmlinux.sh | 10 ++++++++++ > > 1 file changed, 10 insertions(+) > > > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh > > index dd484e92752e..8ddf57cbc439 100755 > > --- a/scripts/link-vmlinux.sh > > +++ b/scripts/link-vmlinux.sh > > @@ -127,6 +127,16 @@ gen_btf() > > cut -d, -f1 | cut -d' ' -f2) > > bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \ > > awk '{print $4}') > > + > > + # Compatibility issues: > > + # - pre-2.25 binutils objcopy doesn't support --dump-section > > + # - llvm-objcopy doesn't support --change-section-address, but > > + # does support --dump-section > > + # > > + # Try to use --dump-section which should cover both recent > > + # binutils and llvm-objcopy and fall back to --only-section > > + # for pre-2.25 binutils. > > + ${OBJCOPY} --dump-section .BTF=$bin_file ${1} 2>/dev/null || \ > > ${OBJCOPY} --change-section-address .BTF=0 \ > > --set-section-flags .BTF=alloc -O binary \ > > --only-section=.BTF ${1} .btf.vmlinux.bin > > -- > > 2.25.1.481.gfbce0eb801-goog > > > > Hi Stanislav, > > Thank you for the patch! This is targeting the bpf tree but it uses the > bin_file variable that comes from commit af73d78bd384 ("kbuild: Remove > debug info from kallsyms linking") in the bpf-next tree. In this form, > when applied to mainline, the first command fails because $bin_file > doesn't exist and falls back to the second command, which results in no > net change for llvm-objcopy. > > When manually applied to next-20200316 or applied to mainline with > $bin_file replaced with .btf.vmlinux.bin, x86_64 and aarch64 successfully > generate BTF with tip of tree llvm-objcopy. > > Tested-by: Nathan Chancellor <natechancellor@gmail.com> Oops, sorry, I did it on top of bpf-next, but then blindly cherry-picked on top of bpf tree. In this case, should we put in only in bpf-next? Do we need a stable backport for the older versions? ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-03-17 3:14 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-03-16 22:25 [PATCH bpf] bpf: Support llvm-objcopy for vmlinux BTF Stanislav Fomichev 2020-03-16 23:15 ` Fangrui Song 2020-03-16 23:56 ` Stanislav Fomichev 2020-03-17 0:56 ` Fangrui Song 2020-03-17 3:14 ` Andrii Nakryiko 2020-03-16 23:57 ` Nathan Chancellor 2020-03-17 0:01 ` Stanislav Fomichev
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).