Re: [PATCH bpf-next] bpf, vmtest: Build test_progs and friends as statically linked

From: Daniel Xu <dxu@dxuuu.xyz>
To: Daniel Borkmann <daniel@iogearbox.net>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>,
	andrii@kernel.org,  bpf@vger.kernel.org
Subject: Re: [PATCH bpf-next] bpf, vmtest: Build test_progs and friends as statically linked
Date: Wed, 12 Jul 2023 10:42:05 -0600	[thread overview]
Message-ID: <m4f3vzvkgt2iov3yvqhdamkkjnoqgqqhhuf3xvrh5cgb4ifohj@g6pybaplatks> (raw)
In-Reply-To: <0456d565-a0d9-8c37-b36a-2d9b2098733d@iogearbox.net>

Hi Daniel,

On Wed, Jul 12, 2023 at 02:38:50PM +0200, Daniel Borkmann wrote:
> On 7/11/23 10:15 PM, Daniel Xu wrote:
> > On Wed, May 31, 2023 at 09:53:57PM +0200, Daniel Borkmann wrote:
> > > On 5/31/23 9:02 PM, Andrii Nakryiko wrote:
> > > > On Fri, May 26, 2023 at 3:47 PM Daniel Borkmann <daniel@iogearbox.net> wrote:
> > > > > 
> > > > > Small fix for vmtest.sh that I've been carrying locally for quite a while
> > > > > now in order to work around the following linker issue:
> > > > > 
> > > > >     # ./vmtest.sh -- ./test_progs -t lsm
> > > > >     [...]
> > > > >     + ip link set lo up
> > > > >     + [ -x /etc/rcS.d/S50-startup ]
> > > > >     + /etc/rcS.d/S50-startup
> > > > >     ./test_progs -t lsm
> > > > >     ./test_progs: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./test_progs)
> > > > >     ./test_progs: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./test_progs)
> > > > >     [    1.356497] ACPI: PM: Preparing to enter system sleep state S5
> > > > >     [    1.358950] reboot: Power down
> > > > >     [...]
> > > > > 
> > > > > With the specified TRUNNER_LDFLAGS out of vmtest to force static linking
> > > > > runners like test_progs/test_maps/etc work just fine.
> > > > 
> > > > Should we make this a command line option to the vmtest.sh script
> > > > instead? I, for one, can't even successfully build on my machine with
> > > > this, probably due to missing some -static library package (though I
> > > > did install libzstd-static). I'm getting:
> > > 
> > > Interesting, in my case it's the other way round, but yeah that could work
> > > as well.
> > 
> > I had the same zstd linker error. This hacky change fixes it:
> > 
> > ```
> > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> > index 9706e7e5e698..c0d8809fd002 100644
> > --- a/tools/testing/selftests/bpf/Makefile
> > +++ b/tools/testing/selftests/bpf/Makefile
> > @@ -31,7 +31,7 @@ CFLAGS += -g -O0 -rdynamic -Wall -Werror $(GENFLAGS) $(SAN_CFLAGS)    \
> >            -I$(CURDIR) -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR)          \
> >            -I$(TOOLSINCDIR) -I$(APIDIR) -I$(OUTPUT)
> >   LDFLAGS += $(SAN_LDFLAGS)
> > -LDLIBS += -lelf -lz -lrt -lpthread
> > +LDLIBS += -lelf -lz -lrt -lpthread -lzstd
> > 
> >   # Silence some warnings when compiled with clang
> >   ifneq ($(LLVM),)
> > ```
> > 
> > Would be good to get some variant of this patch in.
> 
> The above doesn't work for my env, getting same error with adding the -lzstd.
> Btw, did the patch from above work for you?

Your original patch did not work for me (libelf missing zstd symbols).
But with the patch I sent applied, it builds correctly.

What is the error you're getting with `-lzstd`? My guess is that your
distro's libelf.a has zstd symbols statically linked in already. Perhaps
check what pkg-config says about your libelf? Mine is:

```
$ pkg-config --static --libs libelf
-lelf -lz -lzstd
```

Also, perhaps the build should be using pkg-config instead of hardcoding
deps in LDLIBS. That might make the static build more reliable across
different systems.

Thanks,
Daniel