From: Yonghong Song <yhs@fb.com>
To: <sedat.dilek@gmail.com>
Cc: <bpf@vger.kernel.org>, Andrii Nakryiko <andrii@kernel.org>,
Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>,
<kernel-team@fb.com>, Nick Desaulniers <ndesaulniers@google.com>
Subject: Re: [PATCH bpf-next 3/5] selftests/bpf: fix test_cpp compilation failure with clang
Date: Sun, 11 Apr 2021 12:08:13 -0700 [thread overview]
Message-ID: <3f224f2c-bb7e-accc-b095-7fee8210861b@fb.com> (raw)
In-Reply-To: <CA+icZUWwSg4Nd+AzAMx8Os4iAfs=40zeoYn0eVKg3Cy7fB5Cow@mail.gmail.com>
On 4/11/21 10:31 AM, Sedat Dilek wrote:
> On Sun, Apr 11, 2021 at 7:20 PM Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 4/11/21 3:47 AM, Sedat Dilek wrote:
>>> On Sat, Apr 10, 2021 at 6:49 PM Yonghong Song <yhs@fb.com> wrote:
>>>>
>>>> With clang compiler:
>>>> make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
>>>> make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
>>>> the test_cpp build failed due to the failure:
>>>> warning: treating 'c-header' input as 'c++-header' when in C++ mode, this behavior is deprecated [-Wdeprecated]
>>>> clang-13: warning: cannot specify -o when generating multiple output files
>>>>
>>>> test_cpp compilation flag looks like:
>>>> clang++ -g -Og -rdynamic -Wall -I<...> ... \
>>>> -Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program \
>>>> test_cpp.cpp <...>/test_core_extern.skel.h <...>/libbpf.a <...>/test_stub.o \
>>>> -lcap -lelf -lz -lrt -lpthread -o <...>/test_cpp
>>>>
>>>> The clang++ compiler complains the header file in the command line.
>>>> Let us remove the header file from the command line which is not intended
>>>> any way, and this fixed the problem.
>>>>
>>>> Signed-off-by: Yonghong Song <yhs@fb.com>
>>>> ---
>>>> tools/testing/selftests/bpf/Makefile | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
>>>> index 6448c626498f..bbd61cc3889b 100644
>>>> --- a/tools/testing/selftests/bpf/Makefile
>>>> +++ b/tools/testing/selftests/bpf/Makefile
>>>> @@ -481,7 +481,7 @@ $(OUTPUT)/test_verifier: test_verifier.c verifier/tests.h $(BPFOBJ) | $(OUTPUT)
>>>> # Make sure we are able to include and link libbpf against c++.
>>>> $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
>>>> $(call msg,CXX,,$@)
>>>> - $(Q)$(CXX) $(CFLAGS) $^ $(LDLIBS) -o $@
>>>> + $(Q)$(CXX) $(CFLAGS) test_cpp.cpp $(BPFOBJ) $(LDLIBS) -o $@
>>>>
>>>> # Benchmark runner
>>>> $(OUTPUT)/bench_%.o: benchs/bench_%.c bench.h
>>>> --
>>>> 2.30.2
>>>>
>>>
>>> NOTE: bpf-next might be different from my build-environment.
>>>
>>> Yonghong, can you please re-test by adding explicitly CXX=g++ to your make line?
>>>
>>> Here I have:
>>>
>>> $ grep test_cpp make-log_tools-testing-selftests-bpf_clang_clang++.txt
>>> 1907:clang++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/include
>>> -I/home/dileks/src/linux-kernel/git/include/generated
>>> -I/home/dileks/src/linux-kernel/git/tools/lib
>>> -I/home/dileks/src/linux-kernel/git/tools/include
>>> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>> -Wno-unused-command-line-argument -Wno-format-security
>>> -Dbpf_prog_load=bpf_prog_test_load
>>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
>>> -lcap -lelf -lz -lrt -lpthread -o
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
>>>
>>> This clang++ line does not include <...>/test_core_extern.skel.h ^^^
>>>
>>> $ grep test_core_extern.skel.h
>>> make-log_tools-testing-selftests-bpf_clang_clang++.txt
>>> 704:/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/sbin/bpftool
>>> gen skeleton /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_co
>>> re_extern.o > /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_core_extern.skel.h
>>> 1592:/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/sbin/bpftool
>>> gen skeleton /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/no_alu
>>> 32/test_core_extern.o >
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/no_alu32/test_core_extern.skel.h
>>>
>>> Checking test_cpp:
>>>
>>> $ /opt/llvm-toolchain/bin/llvm-objdump-12 -Dr test_cpp | grep extern
>>> 0000000000417e50 <cmp_externs>:
>>> 417e54: 75 22 jne 0x417e78 <cmp_externs+0x28>
>>> 417e59: 75 10 jne 0x417e6b <cmp_externs+0x1b>
>>> 417e61: 75 21 jne 0x417e84 <cmp_externs+0x34>
>>> 417e69: 75 1e jne 0x417e89 <cmp_externs+0x39>
>>> 417e87: eb f2 jmp 0x417e7b <cmp_externs+0x2b>
>>> 417e8c: eb ed jmp 0x417e7b <cmp_externs+0x2b>
>>>
>>> $ /opt/llvm-toolchain/bin/llvm-objdump-12 -Dr test_cpp | grep core_extern
>>> [ EMPTY ]
>>>
>>> When compiled with g++ in an earlier setup this contained "core_extern".
>>>
>>> With this version of your patchser it breaks *here* when using CXX=g++
>>> (and uses /usr/bin/ld as linker):
>>>
>>> g++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/b
>>> pf/tools/include -I/home/dileks/src/linux-kernel/git/include/generated
>>> -I/home/dileks/src/linux-kernel/git/tools/lib
>>> -I/home/dileks/src/linux-kernel/git/tools/include
>>> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>> -Wno-unused-command-line-argument -Wno-format-security
>>> -Dbpf_prog_load=bpf_prog_test_load
>>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
>>> -lcap -lelf -lz -lrt -lpthread -o
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
>>>
>>> /usr/bin/ld: /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a(libbpf-in.o):
>>> relocation R_X86_64_32 against `.rodata.str1.1' ca
>>> n not be used when making a PIE object; recompile with -fPIE
>>> collect2: error: ld returned 1 exit status
>>> make: *** [Makefile:457:
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp]
>>> Error 1
>>
>> I cannot reproduce the issue with g++ with bpf-next, my command line is
>>
>> g++ -g -Og -rdynamic -Wall
>> -I/home/yhs/work/bpf-next/tools/testing/selftests/bpf
>> -I/home/yhs/work/bpf-next/tools/testing/selftests/bpf/tools/include
>> -I/home/yhs/work/bpf-next/include/generated
>> -I/home/yhs/work/bpf-next/tools/lib
>> -I/home/yhs/work/bpf-next/tools/include
>> -I/home/yhs/work/bpf-next/tools/include/uapi
>> -I/home/yhs/work/bpf-next/tools/testing/selftests/bpf
>> -Wno-unused-command-line-argument -Wno-format-security
>> -Dbpf_prog_load=bpf_prog_test_load
>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
>> /home/yhs/work/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
>> -lcap -lelf -lz -lrt -lpthread -o
>> /home/yhs/work/bpf-next/tools/testing/selftests/bpf/test_cpp
>>
>> I modified to
>> g++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
>> -I/home/yhs/work/bpf-next/tools/testing/selftests/bpf ...
>> and cannot reproduce the issue.
>> The macro HAVE_GENHDR is only effect for test_verifier.
>>
>>
>> Could you try to run the above g++ command by adding
>> test_core_extern.skel.h back, something like
>>
>> > g++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
>> > -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>> > -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/b
>> > pf/tools/include -I/home/dileks/src/linux-kernel/git/include/generated
>> > -I/home/dileks/src/linux-kernel/git/tools/lib
>> > -I/home/dileks/src/linux-kernel/git/tools/include
>> > -I/home/dileks/src/linux-kernel/git/tools/include/uapi
>> > -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>> > -Wno-unused-command-line-argument -Wno-format-security
>> > -Dbpf_prog_load=bpf_prog_test_load
>> > -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
>> > test_core_extern.skel.h
>> >
>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
>> > -lcap -lelf -lz -lrt -lpthread -o
>> > /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
>>
>> The issue could be somewhere else?
>>
>
> I have seen all that *skel* was reworked in bpf-next, so this is an issue here.
>
> Adding test_core_extern.skel.h:
>
> $ cd /tools/testing/selftests/bpf/
>
> $ file test_core_extern.skel.h
> test_core_extern.skel.h: C source, ASCII text
>
> $ g++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/include
> -I/home/dileks/src/linux-kernel/git/include/generated
> -I/home/dileks/src/linux-kernel/git/tools/lib
> -I/home/dileks/src/linux-kernel/git/tools/include
> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
> -Wno-unused-command-line-argument -Wno-format-security
> -Dbpf_prog_load=bpf_prog_test_load
> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
> test_core_extern.skel.h
> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
> -lcap -lelf -lz -lrt -lpthread -o
> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
> /usr/bin/ld: /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a(libbpf-in.o):
> relocation R_X86_64_32 against `.rodata.str1.1' ca
> n not be used when making a PIE object; recompile with -fPIE
> collect2: error: ld returned 1 exit status
So the issue is not related to this patch since adding
test_core_extern.skel.h the failure is still there.
So looks like this is g++ compilation issue. So you should be
able to reproduce the issue without this patch set, right?
My gcc compiled libbpf.a also has a bunch of R_X86_64_32 relations
against the .rodata.str1.1 section:
$ readelf -r libbpf.a | less
File: libbpf.a(libbpf-in.o)
Relocation section '.rela.text' at offset 0x11a4b8 contains 3152 entries:
Offset Info Type Sym. Value Sym. Name
+ Addend
000000000136 00020000000b R_X86_64_32S 0000000000000000 .rodata + 0
00000000013b 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + 6e
000000000141 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + c
000000000147 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + 10
00000000014d 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + 16
000000000153 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + 1d
000000000159 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + 23
00000000015f 00030000000a R_X86_64_32 0000000000000000
.rodata.str1.1 + 28
...
and I did not see any issues.
Adding '-###' to the compilation flags you can find the flags for
each subcommand, for linker, I have
/usr/libexec/gcc/x86_64-redhat-linux/8/collect2 -plugin
/usr/libexec/gcc/x86_64-redhat-linux/8/liblto_plugin.so
"-plugin-opt=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper"
"-plugin-opt=-fresolution=/tmp/ccjktcVg.res"
"-plugin-opt=-pass-through=-lgcc_s" "-plugin-opt=-pass-through=-lgcc"
"-plugin-opt=-pass-through=-lc" "-plugin-opt=-pass-through=-lgcc_s"
"-plugin-opt=-pass-through=-lgcc" --build-id --no-add-needed
--eh-frame-hdr "--hash-style=gnu" -m elf_x86_64 -export-dynamic
-dynamic-linker /lib64/ld-linux-x86-64.so.2 -o
/home/yhs/work/bpf-next/tools/testing/selftests/bpf/test_cpp
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crt1.o
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crti.o
/usr/lib/gcc/x86_64-redhat-linux/8/crtbegin.o
-L/usr/lib/gcc/x86_64-redhat-linux/8
-L/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64 -L/lib/../lib64
-L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/8/../../..
/tmp/cc0xqkPi.o
/home/yhs/work/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
-lcap -lelf -lz -lrt -lpthread "-lstdc++" -lm -lgcc_s -lgcc -lc -lgcc_s
-lgcc /usr/lib/gcc/x86_64-redhat-linux/8/crtend.o
/usr/lib/gcc/x86_64-redhat-linux/8/../../../../lib64/crtn.o
COLLECT_GCC_OPTIONS='-g' '-rdynamic' '-Wall' '-O2' '-D' 'HAVE_GENHDR'
'-I' '/home/yhs/work/bpf-next/tools/testing/selftests/bpf' '-I'
'/home/yhs/work/bpf-next/tools/testing/selftests/bpf/tools/include' '-I'
'/home/yhs/work/bpf-next/include/generated' '-I'
'/home/yhs/work/bpf-next/tools/lib' '-I'
'/home/yhs/work/bpf-next/tools/include' '-I'
'/home/yhs/work/bpf-next/tools/include/uapi' '-I'
'/home/yhs/work/bpf-next/tools/testing/selftests/bpf'
'-Wno-uuunused-command-line-argument' '-Wno-format-security' '-D'
'bpf_prog_load=bpf_prog_test_load' '-D'
'bpf_load_program=bpf_test_load_program' '-o'
'/home/yhs/work/bpf-next/tools/testing/selftests/bpf/test_cpp'
'-shared-libgcc' '-mtune=generic' '-march=x86-64'
Not sure whether you will see some different COLLECT_GCC_OPTIONS flags
or not.
There is not much I can do here since I cannot reproduce the issue.
Maybe you can help to narrow down the reason of the failure a little
bit more.
>
> Write that test_cpp.cpp in C :-)?
test_cpp.cpp is intentionally to test libbpf used by a c++ application.
>
> BTW, did you check (llvm-)objdump output?
>
> $ /opt/llvm-toolchain/bin/llvm-objdump-12 -Dr test_cpp | grep core_extern
This is what I got with g++ compiled test_cpp:
$ llvm-objdump -Dr test_cpp | grep core_extern
406a80: e8 5b 01 00 00 callq 0x406be0
<_ZL25test_core_extern__destroyP16test_core_extern>
406ab9: e8 22 01 00 00 callq 0x406be0
<_ZL25test_core_extern__destroyP16test_core_extern>
0000000000406be0 <_ZL25test_core_extern__destroyP16test_core_extern>:
406be3: 74 1a je 0x406bff
<_ZL25test_core_extern__destroyP16test_core_extern+0x1f>
406bef: 74 05 je 0x406bf6
<_ZL25test_core_extern__destroyP16test_core_extern+0x16>
>
> - Sedat -
>
>>>
>>> $ grep test_cpp ../make-log_tools-testing-selftests-bpf_clang_g++.txt
>>> | grep test_core_extern.skel.h
>>> [ EMPTY ]
>>>
>>> As said I do NOT use bpf-next.
>>>
>>> - Sedat -
>>>
>>>
>>>
>>> - Sedat -
>>>
next prev parent reply other threads:[~2021-04-11 19:08 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-10 16:49 [PATCH bpf-next 0/5] support build selftests/bpf with clang Yonghong Song
2021-04-10 16:49 ` [PATCH bpf-next 1/5] selftests: set CC to clang in lib.mk if LLVM is set Yonghong Song
2021-04-11 10:22 ` Sedat Dilek
2021-04-11 16:51 ` Yonghong Song
2021-04-11 17:10 ` Sedat Dilek
2021-04-10 16:49 ` [PATCH bpf-next 2/5] tools: allow proper CC/CXX/... override with LLVM=1 in Makefile.include Yonghong Song
2021-04-11 10:24 ` Sedat Dilek
2021-04-11 16:52 ` Yonghong Song
2021-04-10 16:49 ` [PATCH bpf-next 3/5] selftests/bpf: fix test_cpp compilation failure with clang Yonghong Song
2021-04-11 10:47 ` Sedat Dilek
2021-04-11 17:20 ` Yonghong Song
2021-04-11 17:31 ` Sedat Dilek
2021-04-11 19:08 ` Yonghong Song [this message]
2021-04-12 4:47 ` Sedat Dilek
2021-04-12 5:42 ` Yonghong Song
2021-04-12 6:06 ` Sedat Dilek
2021-04-12 14:15 ` Yonghong Song
2021-04-13 4:32 ` Andrii Nakryiko
2021-04-13 6:12 ` Yonghong Song
2021-04-13 17:50 ` Andrii Nakryiko
2021-04-10 16:49 ` [PATCH bpf-next 4/5] selftests/bpf: silence clang compilation warnings Yonghong Song
2021-04-11 11:12 ` Sedat Dilek
2021-04-11 17:40 ` Yonghong Song
2021-04-10 16:49 ` [PATCH bpf-next 5/5] bpftool: fix a clang compilation warning Yonghong Song
2021-04-11 11:05 ` Sedat Dilek
2021-04-11 17:24 ` Yonghong Song
2021-04-10 17:23 ` [PATCH bpf-next 0/5] support build selftests/bpf with clang Alexei Starovoitov
2021-04-10 17:38 ` Yonghong Song
2021-04-10 19:19 ` Sedat Dilek
2021-04-11 16:46 ` Yonghong Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3f224f2c-bb7e-accc-b095-7fee8210861b@fb.com \
--to=yhs@fb.com \
--cc=andrii@kernel.org \
--cc=arnaldo.melo@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=kernel-team@fb.com \
--cc=ndesaulniers@google.com \
--cc=sedat.dilek@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).