On Fri, 2021-07-16 at 22:12 +0200, Michal Suchánek wrote:
> On Fri, Jul 16, 2021 at 08:08:27PM +0100, Luca Boccassi wrote:
> > On Fri, 2021-07-16 at 14:35 +0100, Luca Boccassi wrote:
> > > On Fri, 2021-07-16 at 10:25 -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Thu, Jul 15, 2021 at 11:31:20PM +0200, Michal Suchánek escreveu:
> > > > > Hello,
> > > > > 
> > > > > when building with system libbpf I get:
> > > > > 
> > > > > [   40s] make[1]: Nothing to be done for 'preinstall'.
> > > > > [   40s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/dwarves-1.21+git175.1ef87b2/build'
> > > > > [   40s] Install the project...
> > > > > [   40s] /usr/bin/cmake -P cmake_install.cmake
> > > > > [   40s] -- Install configuration: "RelWithDebInfo"
> > > > > [   40s] -- Installing: /home/abuild/rpmbuild/BUILDROOT/dwarves-1.21+git175.1ef87b2-15.1.ppc64le/usr/bin/codiff
> > > > > [   40s] CMake Error at cmake_install.cmake:63 (file):
> > > > > [   40s]   file RPATH_CHANGE could not write new RPATH:
> > > > > [   40s] 
> > > > > [   40s]     
> > > > > [   40s] 
> > > > > [   40s]   to the file:
> > > > > [   40s] 
> > > > > [   40s]     /home/abuild/rpmbuild/BUILDROOT/dwarves-1.21+git175.1ef87b2-15.1.ppc64le/usr/bin/codiff
> > > > > [   40s] 
> > > > > [   40s]   The current RUNPATH is:
> > > > > [   40s] 
> > > > > [   40s]     /home/abuild/rpmbuild/BUILD/dwarves-1.21+git175.1ef87b2/build:
> > > > > [   40s] 
> > > > > [   40s]   which does not contain:
> > > > > [   40s] 
> > > > > [   40s]     /usr/local/lib64:/home/abuild/rpmbuild/BUILD/dwarves-1.21+git175.1ef87b2/build:
> > > > > [   40s] 
> > > > > [   40s]   as was expected.
> > > > > [   40s] 
> > > > > [   40s] 
> > > > > [   40s] make: *** [Makefile:74: install] Error 1
> > > > > [   40s] make: Leaving directory '/home/abuild/rpmbuild/BUILD/dwarves-1.21+git175.1ef87b2/build'
> > > > > [   40s] error: Bad exit status from /var/tmp/rpm-tmp.OaGNX4 (%install)
> > > > > 
> > > > > This is not a problem with embedded libbpf.
> > > > > 
> > > > > Using system libbpf seems to be new in 1.22
> > > > 
> > > > Lucca, can you take a look at this?
> > > > 
> > > > Thanks,
> > > > 
> > > > - Arnaldo
> > > 
> > > Hi,
> > > 
> > > Michal, what is the CMake configuration command line you are using?
> > 
> > Latest tmp.master builds fine here with libbpf 0.4.0. To reproduce it
> > would be good to know build env and command line used, otherwise I
> > can't really tell.
> 
> Indeed, there is non-trivial rpm macro expanded when invoking cmake.
> 
> Attaching log.
> 
> Thanks
> 
> Michal

So, this took some spelunking. TL;DR: openSUSE's libbpf.pc from libbpf-
devel is broken, it lists -L/usr/local/lib64 in Libs even though it
doesn't install anything in that prefix. Fix it to list the right path
and the problem goes away.

Longer version: CMake is complaining that the rpath in the binary is
not what it expected it to be when stripping it. Of course the first
question is, why does that matter since it's removing it? Just remove
it, no? Another CMake weirdness to add the the list, I guess.

[   20s]   file RPATH_CHANGE could not write new RPATH:
[   20s] 
[   20s]     
[   20s] 
[   20s]   to the file:
[   20s] 
[   20s]     /home/abuild/rpmbuild/BUILDROOT/dwarves-
1.21+git175.1ef87b2-21.1.x86_64/usr/bin/codiff
[   20s] 
[   20s]   The current RUNPATH is:
[   20s] 
[   20s]     /home/abuild/rpmbuild/BUILD/dwarves-
1.21+git175.1ef87b2/build:
[   20s] 
[   20s]   which does not contain:
[   20s] 
[   20s]     /usr/local/lib64:/home/abuild/rpmbuild/BUILD/dwarves-
1.21+git175.1ef87b2/build:
[   20s] 
[   20s]   as was expected.

This is the linking step where the rpath is set:

[   19s] /usr/bin/cc -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-
strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-
protection -Werror=return-type -flto=auto -g -DNDEBUG
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -DNDEBUG -flto=auto
-Wl,--as-needed -Wl,--no-undefined -Wl,-z,now -rdynamic
CMakeFiles/codiff.dir/codiff.c.o -o codiff   -L/usr/local/lib64  -Wl,-
rpath,/usr/local/lib64:/home/abuild/rpmbuild/BUILD/dwarves-
1.21+git175.1ef87b2/build: libdwarves.so.1.0.0 -ldw -lelf -lz -lbpf 

On a build without -DLIBBPF_EMBEDDED=off, this is the linking step for
the same binary:

[   64s] /usr/bin/cc -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-
strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-
protection -Werror=return-type -flto=auto -g -DNDEBUG
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -DNDEBUG -flto=auto
-Wl,--as-needed -Wl,--no-undefined -Wl,-z,now -rdynamic
CMakeFiles/codiff.dir/codiff.c.o -o codiff  -Wl,-
rpath,/home/abuild/rpmbuild/BUILD/dwarves-1.21+git175.1ef87b2/build:
libdwarves.so.1.0.0 -ldw -lelf -lz

/usr/local/lib64 is not in the rpath. Why? The hint is that
-L/usr/local/lib64 is not passed either, too much of a coincidence to
be unrelated.

Where is it coming from? Well, when using the system's libbpf we are
using the pkgconfig file from the package. It is common to list linker
flags in there, although this one shouldn't be in it. Sure enough,
downloading libbpf-devel from 
https://build.opensuse.org/package/binaries/openSUSE:Factory/libbpf/standard
and extracting the pc file:

$ cat /tmp/libbpf.pc 
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)

prefix=/usr/local
libdir=/usr/local/lib64
includedir=${prefix}/include

Name: libbpf
Description: BPF library
Version: 0.3.0
Libs: -L${libdir} -lbpf
Requires.private: libelf zlib
Cflags: -I${includedir}

There it is. Nothing is installed in that path, so it shouldn't be
there in the first place.

$ rpm -qlp /tmp/libbpf0-5.12.4-2.7.x86_64.rpm 
warning: /tmp/libbpf0-5.12.4-2.7.x86_64.rpm: Header V3 RSA/SHA256
Signature, key ID 3dbdc284: NOKEY
/usr/lib64/libbpf.so.0
/usr/lib64/libbpf.so.0.3.0
$ rpm -qlp /tmp/libbpf-devel-5.12.4-2.7.x86_64.rpm 
warning: /tmp/libbpf-devel-5.12.4-2.7.x86_64.rpm: Header V3 RSA/SHA256
Signature, key ID 3dbdc284: NOKEY
/usr/include/bpf
/usr/include/bpf/bpf.h
/usr/include/bpf/bpf_core_read.h
/usr/include/bpf/bpf_endian.h
/usr/include/bpf/bpf_helper_defs.h
/usr/include/bpf/bpf_helpers.h
/usr/include/bpf/bpf_tracing.h
/usr/include/bpf/btf.h
/usr/include/bpf/libbpf.h
/usr/include/bpf/libbpf_common.h
/usr/include/bpf/libbpf_util.h
/usr/include/bpf/xsk.h
/usr/lib64/libbpf.so
/usr/lib64/pkgconfig/libbpf.pc

After hacking it out in the libbpf build:

https://build.opensuse.org/package/show/home:bluca:branches:home:michals/libbpf

Dwarves then builds just fine with system libbpf:

https://build.opensuse.org/package/show/home:bluca:branches:home:michals/dwarves

Here's a PR for the libbpf package that applies a quick hack to fix it:

https://build.opensuse.org/request/show/906834

I'll send my invoice to SUSE Inc. ;-)

-- 
Kind regards,
Luca Boccassi