Re: [PATCH dwarves] dwarf_loader: handle DWARF5 DW_OP_addrx properly

[Yonghong Song <yhs@fb.com>]

On 4/5/21 11:32 AM, Sedat Dilek wrote:
> On Mon, Apr 5, 2021 at 6:17 PM Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 4/4/21 11:55 PM, Sedat Dilek wrote:
>>> On Mon, Apr 5, 2021 at 4:24 AM Yonghong Song <yhs@fb.com> wrote:
>>>>
>>>>
>>>>
>>>> On 4/4/21 10:25 AM, Sedat Dilek wrote:
>>>>> On Sun, Apr 4, 2021 at 6:40 PM Yonghong Song <yhs@fb.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 4/4/21 5:46 AM, Sedat Dilek wrote:
>>>> [...]
>>>>>>> Next build-error:
>>>>>>>
>>>>>>> g++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
>>>>>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>>>>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/b
>>>>>>> pf/tools/include -I/home/dileks/src/linux-kernel/git/include/generated
>>>>>>> -I/home/dileks/src/linux-kernel/git/tools/lib
>>>>>>> -I/home/dileks/src/linux-kernel/git/tools/include
>>>>>>> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
>>>>>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>>>>>> -Dbpf_prog_load=bpf_prog_test_load
>>>>>>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
>>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_core_extern.skel.h
>>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
>>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_stub.o
>>>>>>> -lcap -lelf -lz -lrt -lpthread -o
>>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
>>>>>>> /usr/bin/ld: /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a(libbpf-in.o):
>>>>>>> relocation R_X86_64_32 against `.rodata.str1.1' ca
>>>>>>> n not be used when making a PIE object; recompile with -fPIE
>>>>>>> collect2: error: ld returned 1 exit status
>>>>>>> make: *** [Makefile:455:
>>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp]
>>>>>>> Error 1
>>>>>>> make: Leaving directory
>>>>>>> '/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf'
>>>>>>>
>>>>>>> LOL, I was not aware that there is usage of *** CXX*** in tools
>>>>>>> directory (see g++ line and /usr/bin/ld ?).
>>>>>>>
>>>>>>> So, I changed my $MAKE_OPTS to use "CXX=clang++".
>>>>>>
>>>>>> In kernel, if LLVM=1 is set, we have:
>>>>>>
>>>>>> ifneq ($(LLVM),)
>>>>>> HOSTCC  = clang
>>>>>> HOSTCXX = clang++
>>>>>> else
>>>>>> HOSTCC  = gcc
>>>>>> HOSTCXX = g++
>>>>>> endif
>>>>>>
>>>>>> ifneq ($(LLVM),)
>>>>>> CC              = clang
>>>>>> LD              = ld.lld
>>>>>> AR              = llvm-ar
>>>>>> NM              = llvm-nm
>>>>>> OBJCOPY         = llvm-objcopy
>>>>>> OBJDUMP         = llvm-objdump
>>>>>> READELF         = llvm-readelf
>>>>>> STRIP           = llvm-strip
>>>>>> else
>>>>>> CC              = $(CROSS_COMPILE)gcc
>>>>>> LD              = $(CROSS_COMPILE)ld
>>>>>> AR              = $(CROSS_COMPILE)ar
>>>>>> NM              = $(CROSS_COMPILE)nm
>>>>>> OBJCOPY         = $(CROSS_COMPILE)objcopy
>>>>>> OBJDUMP         = $(CROSS_COMPILE)objdump
>>>>>> READELF         = $(CROSS_COMPILE)readelf
>>>>>> STRIP           = $(CROSS_COMPILE)strip
>>>>>> endif
>>>>>>
>>>>>> So if you have right path, you don't need to set HOSTCC and HOSTCXX
>>>>>> explicitly.
>>>>>>
>>>>>
>>>>> That is all correct with HOSTCXX but there is no CXX=... assignment
>>>>> otherwise test_cpp will use g++ as demonstrated.
>>>>
>>>> This is not a kernel Makefile issue.
>>>>
>>>> We have:
>>>> testing/selftests/bpf/Makefile:CXX ?= $(CROSS_COMPILE)g++
>>>>
>>>> So you need to explicit add CXX=clang++ when compiling
>>>> bpf selftests with LLVM=1 LLVM_IAS=1.
>>>>
>>>
>>> NOPE.
>>>
>>> $ echo $MAKE $MAKE_OPTS
>>> make V=1 LLVM=1 LLVM_IAS=1 CXX=clang++ PAHOLE=/opt/pahole/bin/pahole
>>>
>>> $ LC_ALL=C $MAKE $MAKE_OPTS -C tools/testing/selftests/bpf/ 2>&1 | tee
>>> ../make-log_tools-testing-selftests-bpf_llvm-1-llvm_ias-1_cxx-clang.txt
>>>
>>> This breaks again like reported before:
>>>
>>> clang++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/include
>>> -I/home/dileks/src/linux-kernel/git/include/generated
>>> -I/home/dileks/src/linux-kernel/git/tools/lib
>>> -I/home/dileks/src/linux-kernel/git/tools/include
>>> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
>>> -Dbpf_prog_load=bpf_prog_test_load
>>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_core_extern.skel.h
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_stub.o
>>> -lcap -lelf -lz -lrt -lpthread -o
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
>>>
>>> clang-12: warning: treating 'c-header' input as 'c++-header' when in
>>> C++ mode, this behavior is deprecated [-Wdeprecated]
>>> clang-12: error: cannot specify -o when generating multiple output files
>>> make: *** [Makefile:455:
>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp]
>>> Error 1
>>> make: Leaving directory
>>> '/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf'
>>>
>>> Do you know some magic CXX flags to be passed?
>>
>> I tested in my environment. The reason is LC_ALL=C.
>> Without LC_ALL=C, make succeeded and with it, test_cpp
>> compilation failed. Is it possible for you to drop
>> LC_ALL=C for bpf selftests?
>>
>> The following command succeeded for me:
>>      make -C tools/testing/selftests/bpf -j60 LLVM=1 V=1 CXX=clang++ CC=clang
>>
> 
> First, I tried the exact make invocation ^^^ in my build-environment
> but that breaks with the same ERROR.
> 
> I did in a second run:
> 
> LLVM_TOOLCHAIN_PATH="/opt/llvm-toolchain/bin"
> if [ -d ${LLVM_TOOLCHAIN_PATH} ]; then
>    export PATH="${LLVM_TOOLCHAIN_PATH}:${PATH}"
> fi
> 
> echo $PATH
> /opt/llvm-toolchain/bin:/opt/proxychains-ng/bin:/home/dileks/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
> 
> MAKE="make"
> MAKE_OPTS="V=1 -j4 LLVM=1 CC=clang CXX=clang++"
> MAKE_OPTS="$MAKE_OPTS PAHOLE=/opt/pahole/bin/pahole"
> 
> echo $MAKE $MAKE_OPTS
> make V=1 -j4 LLVM=1 CC=clang CXX=clang++ PAHOLE=/opt/pahole/bin/pahole
> 
> $MAKE $MAKE_OPTS -C tools/testing/selftests/bpf/ 2>&1 | tee
> ../make-log_tools-testing-selftests-bpf.txt
> 
> That would have been funny... Drop LC_ALL=C from make line as a fix.
> 
> Just curious: Do you see these warnings?
> 
> clang-12: warning: argument unused during compilation: '-rdynamic'
> [-Wunused-command-line-argument]
> clang-12: warning: -lcap: 'linker' input unused [-Wunused-command-line-argument]
> clang-12: warning: -lelf: 'linker' input unused [-Wunused-command-line-argument]
> clang-12: warning: -lz: 'linker' input unused [-Wunused-command-line-argument]
> clang-12: warning: -lrt: 'linker' input unused [-Wunused-command-line-argument]
> clang-12: warning: -lpthread: 'linker' input unused
> [-Wunused-command-line-argument]
> clang-12: warning: -lm: 'linker' input unused [-Wunused-command-line-argument]
> 
> Equivalent CFLAGS for '-rdynamic' when CC=clang is used?
> Missing LDFLAGS when LD=ld.lld (make LLVM=1) is used?

I see this warning as well, but seems it does not hurt...
Maybe some flags need change if the CC is clang...

> 
> Last question:
> Can you pass LLVM_IAS=1 (means use LLVM/Clang Integrated ASsembler) to
> your make line?
> 
> Old: make -C tools/testing/selftests/bpf -j60 LLVM=1 V=1 CXX=clang++ CC=clang
> New: make -C tools/testing/selftests/bpf -j60 LLVM=1 LLVM_IAS=1 V=1
> CXX=clang++ CC=clang
> 
> Does it build successfully?

I think it is better to add LLVM_IAS=1. In my build, for non-lto, I 
didn't pass LLVM_IAS=1 to kernel build, so selftest does not need it either.

But for LTO build, LLVM_IAS=1 is required so selftest also needs
LLVM_IAS=1.

Yes, we can always have LLVM_IAS=1 if it is included in kernel build.

> 
> - Sedat -
> 
>>>
>>> The only solution is to suppress the build of test_cpp (see
>>> TEST_GEN_PROGS_EXTENDED):
>>>
>>> $ git diff tools/testing/selftests/bpf/Makefile
>>> diff --git a/tools/testing/selftests/bpf/Makefile
>>> b/tools/testing/selftests/bpf/Makefile
>>> index 044bfdcf5b74..cf7c7c8f72cf 100644
>>> --- a/tools/testing/selftests/bpf/Makefile
>>> +++ b/tools/testing/selftests/bpf/Makefile
>>> @@ -77,8 +77,8 @@ TEST_PROGS_EXTENDED := with_addr.sh \
>>> # Compile but not part of 'make run_tests'
>>> TEST_GEN_PROGS_EXTENDED = test_sock_addr test_skb_cgroup_id_user \
>>>          flow_dissector_load test_flow_dissector test_tcp_check_syncookie_user \
>>> -       test_lirc_mode2_user xdping test_cpp runqslower bench bpf_testmod.ko \
>>> -       xdpxceiver
>>> +       test_lirc_mode2_user xdping runqslower bench bpf_testmod.ko xdpxceiver
>>> +       # test_cpp # Suppress the build when CXX=clang++ is used
>>>
>>> TEST_CUSTOM_PROGS = $(OUTPUT)/urandom_read
>>>
>>> I have attached both make-logs with and without suppressing the build
>>> of test_cpp and the diff.
>>>
>>> - Sedat -
>>>
>>>>
>>>>>
>> [...]