kernel bpf test_progs - vm wrong libc version

kernel bpf test_progs - vm wrong libc version

[Geyslan G. Bem]

Trying to run vmtest.sh from the bpf-next linux
tools/testing/selftests/bpf on Arch Linux raises this error:

./test_progs
./test_progs: /usr/lib/libc.so.6: version `GLIBC_2.33' not found
(required by ./test_progs)

VM:
https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/libbpf-vmtest-rootfs-2020.09.27.tar.zst

[root@(none) /]# strings /usr/lib/libc.so.6 | grep '^GLIBC_2.' | tail
GLIBC_2.30
GLIBC_2.5
GLIBC_2.9
GLIBC_2.7
GLIBC_2.6
GLIBC_2.18
GLIBC_2.11
GLIBC_2.16
GLIBC_2.13
GLIBC_2.2.6

It would be nice to have
https://github.com/libbpf/libbpf/blob/master/travis-ci/vmtest/configs/INDEX
updated to refer to a new image with GLIBC_2.33.

Host settings:

$ strings /usr/lib/libc.so.6 | grep GLIBC_2.33
GLIBC_2.33
GLIBC_2.33

$ uname -a
Linux hb 5.12.9-arch1-1 #1 SMP PREEMPT Thu, 03 Jun 2021 11:36:13 +0000
x86_64 GNU/Linux

$ gcc --version
gcc (GCC) 11.1.0

$ clang --version
clang version 13.0.0 (/home/uzu/.cache/yay/llvm-git/llvm-project
ad381e39a52604ba07e1e027e7bdec1c287d9089)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

P.S.: This issue was started in
https://github.com/libbpf/libbpf/issues/321 and brought to here.

Thank you.

Regards,

Geyslan G. Bem

Re: kernel bpf test_progs - vm wrong libc version

[Andrii Nakryiko]

On Fri, Jun 11, 2021 at 1:23 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
>
> Trying to run vmtest.sh from the bpf-next linux
> tools/testing/selftests/bpf on Arch Linux raises this error:
>
> ./test_progs
> ./test_progs: /usr/lib/libc.so.6: version `GLIBC_2.33' not found
> (required by ./test_progs)
>
> VM:
> https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/libbpf-vmtest-rootfs-2020.09.27.tar.zst
>
> [root@(none) /]# strings /usr/lib/libc.so.6 | grep '^GLIBC_2.' | tail
> GLIBC_2.30
> GLIBC_2.5
> GLIBC_2.9
> GLIBC_2.7
> GLIBC_2.6
> GLIBC_2.18
> GLIBC_2.11
> GLIBC_2.16
> GLIBC_2.13
> GLIBC_2.2.6
>
> It would be nice to have
> https://github.com/libbpf/libbpf/blob/master/travis-ci/vmtest/configs/INDEX
> updated to refer to a new image with GLIBC_2.33.
>
> Host settings:
>
> $ strings /usr/lib/libc.so.6 | grep GLIBC_2.33
> GLIBC_2.33
> GLIBC_2.33
>

It seems kind of silly to update our perfectly working image just
because a new version of glibc was released. Is there any way for you
to down-grade glibc or build it in some compatibility mode, etc?
selftests don't really rely on any bleeding-edge features of glibc.

> $ uname -a
> Linux hb 5.12.9-arch1-1 #1 SMP PREEMPT Thu, 03 Jun 2021 11:36:13 +0000
> x86_64 GNU/Linux
>
> $ gcc --version
> gcc (GCC) 11.1.0
>
> $ clang --version
> clang version 13.0.0 (/home/uzu/.cache/yay/llvm-git/llvm-project
> ad381e39a52604ba07e1e027e7bdec1c287d9089)
> Target: x86_64-pc-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
>
> P.S.: This issue was started in
> https://github.com/libbpf/libbpf/issues/321 and brought to here.
>
> Thank you.
>
> Regards,
>
> Geyslan G. Bem

Re: kernel bpf test_progs - vm wrong libc version

[Jussi Maki]

On Tue, Jun 15, 2021 at 8:28 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
> It seems kind of silly to update our perfectly working image just
> because a new version of glibc was released. Is there any way for you
> to down-grade glibc or build it in some compatibility mode, etc?
> selftests don't really rely on any bleeding-edge features of glibc.

I've also hit this issue as Ubuntu 21.04 ships with glibc 2.33. I
ended up solving it the hard way by rebuilding the image (I needed few
other tools at the time anyway). Definitely agree it's a bit silly if
we'd need to bump the image every time there's a new glibc version out
there. I did try and see if there's a way to build against newer
glibc, but target older versions and I didn't find a way to do that.
Would statically linking test-progs be an option to avoid this kind of
breakage in the future?

Re: kernel bpf test_progs - vm wrong libc version

[KP Singh]

On Tue, Jun 15, 2021 at 10:06 AM Jussi Maki <joamaki@gmail.com> wrote:
>
> On Tue, Jun 15, 2021 at 8:28 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> > It seems kind of silly to update our perfectly working image just
> > because a new version of glibc was released. Is there any way for you
> > to down-grade glibc or build it in some compatibility mode, etc?
> > selftests don't really rely on any bleeding-edge features of glibc.
>
> I've also hit this issue as Ubuntu 21.04 ships with glibc 2.33. I
> ended up solving it the hard way by rebuilding the image (I needed few
> other tools at the time anyway). Definitely agree it's a bit silly if
> we'd need to bump the image every time there's a new glibc version out
> there. I did try and see if there's a way to build against newer
> glibc, but target older versions and I didn't find a way to do that.
> Would statically linking test-progs be an option to avoid this kind of
> breakage in the future?

I think static linking tests_progs is the only real way one can solve this.
Even if we keep updating the image, there will still be users that will hit
glibc version issues.

Andrii, Maybe we can have a mode for vmtest.sh can build test_progs
statically?

maybe something like:

git diff
diff --git a/tools/testing/selftests/bpf/Makefile
b/tools/testing/selftests/bpf/Makefile
index f405b20c1e6c..5ab0b7e6a95d 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -450,7 +450,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
                             $(RESOLVE_BTFIDS)                          \
                             | $(TRUNNER_BINARY)-extras
        $$(call msg,BINARY,,$$@)
-       $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@
+       $(Q)$$(CC) $$(CFLAGS) $(TRUNNER_LDFLAGS) $$(filter %.a
%.o,$$^) $$(LDLIBS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --no-fail --btf $(TRUNNER_OUTPUT)/btf_data.o $$@

 endef
diff --git a/tools/testing/selftests/bpf/vmtest.sh
b/tools/testing/selftests/bpf/vmtest.sh
index 8889b3f55236..331072074014 100755
--- a/tools/testing/selftests/bpf/vmtest.sh
+++ b/tools/testing/selftests/bpf/vmtest.sh
@@ -138,7 +138,7 @@ update_selftests()
        local selftests_dir="${kernel_checkout}/tools/testing/selftests/bpf"

        cd "${selftests_dir}"
-       ${make_command}
+       TRUNNER_LDFLAGS=-static ${make_command}

        # Mount the image and copy the selftests to the image.
        mount_image

file test_progs
test_progs: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux),
statically linked,
BuildID[sha1]=b1915565b344c412f3eaa07eef4bfdd4a0fc672e, for GNU/Linux
3.2.0, with debug_info, not stripped

I tried with simply doing LDFLAGS=-static but that breaks other binaries:

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginT.o: relocation
R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when
making a shared object
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:167:
/home/kpsingh/projects/krsi/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.so.0.5.0]
Error 1
make[1]: *** [Makefile:135: all] Error 2
make: *** [Makefile:228:
/home/kpsingh/projects/krsi/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a]
Error 2
make: *** Deleting file
'/home/kpsingh/projects/krsi/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a'

Let me know if this works for you.




-

Re: kernel bpf test_progs - vm wrong libc version

[Geyslan G. Bem]

On Tue, 15 Jun 2021 at 06:58, KP Singh <kpsingh@kernel.org> wrote:
>
> On Tue, Jun 15, 2021 at 10:06 AM Jussi Maki <joamaki@gmail.com> wrote:
> >
> > On Tue, Jun 15, 2021 at 8:28 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > > It seems kind of silly to update our perfectly working image just
> > > because a new version of glibc was released. Is there any way for you
> > > to down-grade glibc or build it in some compatibility mode, etc?
> > > selftests don't really rely on any bleeding-edge features of glibc.
> >
> > I've also hit this issue as Ubuntu 21.04 ships with glibc 2.33. I
> > ended up solving it the hard way by rebuilding the image (I needed few
> > other tools at the time anyway). Definitely agree it's a bit silly if
> > we'd need to bump the image every time there's a new glibc version out
> > there. I did try and see if there's a way to build against newer
> > glibc, but target older versions and I didn't find a way to do that.
> > Would statically linking test-progs be an option to avoid this kind of
> > breakage in the future?
>
> I think static linking tests_progs is the only real way one can solve this.
> Even if we keep updating the image, there will still be users that will hit
> glibc version issues.

I agree once the image remains static.

>
> Andrii, Maybe we can have a mode for vmtest.sh can build test_progs
> statically?
>
> maybe something like:

These changes generates the output:

  BINARY   test_maps
/usr/bin/ld: cannot find -lcap
collect2: error: ld returned 1 exit status
make: *** [Makefile:492:
/home/uzu/code/bpf-next/tools/testing/selftests/bpf/test_maps] Error 1

libcap and acl are installed

$ ld --version
GNU ld (GNU Binutils) 2.36.1

$ ld.lld --version
LLD 13.0.0 (/home/uzu/.cache/yay/llvm-git/llvm-project
ad381e39a52604ba07e1e027e7bdec1c287d9089) (compatible with GNU
linkers)

>
> git diff
> diff --git a/tools/testing/selftests/bpf/Makefile
> b/tools/testing/selftests/bpf/Makefile
> index f405b20c1e6c..5ab0b7e6a95d 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -450,7 +450,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
>                  \
>                              $(RESOLVE_BTFIDS)                          \
>                              | $(TRUNNER_BINARY)-extras
>         $$(call msg,BINARY,,$$@)
> -       $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@
> +       $(Q)$$(CC) $$(CFLAGS) $(TRUNNER_LDFLAGS) $$(filter %.a
> %.o,$$^) $$(LDLIBS) -o $$@
>         $(Q)$(RESOLVE_BTFIDS) --no-fail --btf $(TRUNNER_OUTPUT)/btf_data.o $$@
>
>  endef
> diff --git a/tools/testing/selftests/bpf/vmtest.sh
> b/tools/testing/selftests/bpf/vmtest.sh
> index 8889b3f55236..331072074014 100755
> --- a/tools/testing/selftests/bpf/vmtest.sh
> +++ b/tools/testing/selftests/bpf/vmtest.sh
> @@ -138,7 +138,7 @@ update_selftests()
>         local selftests_dir="${kernel_checkout}/tools/testing/selftests/bpf"
>
>         cd "${selftests_dir}"
> -       ${make_command}
> +       TRUNNER_LDFLAGS=-static ${make_command}
>
>         # Mount the image and copy the selftests to the image.
>         mount_image
>
> file test_progs
> test_progs: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux),
> statically linked,
> BuildID[sha1]=b1915565b344c412f3eaa07eef4bfdd4a0fc672e, for GNU/Linux
> 3.2.0, with debug_info, not stripped
>
> I tried with simply doing LDFLAGS=-static but that breaks other binaries:

Earlier I tried just that too. Same results.

>
> /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginT.o: relocation
> R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when
> making a shared object
> collect2: error: ld returned 1 exit status
> make[2]: *** [Makefile:167:
> /home/kpsingh/projects/krsi/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.so.0.5.0]
> Error 1
> make[1]: *** [Makefile:135: all] Error 2
> make: *** [Makefile:228:
> /home/kpsingh/projects/krsi/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a]
> Error 2
> make: *** Deleting file
> '/home/kpsingh/projects/krsi/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a'
>
> Let me know if this works for you.
>
>
>
>
> -



-- 
Regards,

Geyslan G. Bem

Re: kernel bpf test_progs - vm wrong libc version

[KP Singh]

On Tue, Jun 15, 2021 at 4:57 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
>
> On Tue, 15 Jun 2021 at 11:33, KP Singh <kpsingh@kernel.org> wrote:
> >
> > On Tue, Jun 15, 2021 at 2:34 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> > >
> > > On Tue, 15 Jun 2021 at 06:58, KP Singh <kpsingh@kernel.org> wrote:
> > > >
> > > > On Tue, Jun 15, 2021 at 10:06 AM Jussi Maki <joamaki@gmail.com> wrote:
> > > > >
> > > > > On Tue, Jun 15, 2021 at 8:28 AM Andrii Nakryiko
> > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > > It seems kind of silly to update our perfectly working image just
> > > > > > because a new version of glibc was released. Is there any way for you
> > > > > > to down-grade glibc or build it in some compatibility mode, etc?
> > > > > > selftests don't really rely on any bleeding-edge features of glibc.
> > > > >
> > > > > I've also hit this issue as Ubuntu 21.04 ships with glibc 2.33. I
> > > > > ended up solving it the hard way by rebuilding the image (I needed few
> > > > > other tools at the time anyway). Definitely agree it's a bit silly if
> > > > > we'd need to bump the image every time there's a new glibc version out
> > > > > there. I did try and see if there's a way to build against newer
> > > > > glibc, but target older versions and I didn't find a way to do that.
> > > > > Would statically linking test-progs be an option to avoid this kind of
> > > > > breakage in the future?
> > > >
> > > > I think static linking tests_progs is the only real way one can solve this.
> > > > Even if we keep updating the image, there will still be users that will hit
> > > > glibc version issues.
> > >
> > > I agree once the image remains static.
> > >
> > > >
> > > > Andrii, Maybe we can have a mode for vmtest.sh can build test_progs
> > > > statically?
> > > >
> > > > maybe something like:
> > >
> > > These changes generates the output:
> > >
> > >   BINARY   test_maps
> > > /usr/bin/ld: cannot find -lcap
> > > collect2: error: ld returned 1 exit status
> > > make: *** [Makefile:492:
> > > /home/uzu/code/bpf-next/tools/testing/selftests/bpf/test_maps] Error 1
> > >
> > > libcap and acl are installed
> >
> > Are you sure you have libcap-dev installed? I don't see this on my system.
>
> As Arch packages maintain headers, I suppose libcap has everything.
>
> $ yay -F libcap.so
> core/libcap 2.49-1 [installed: 2.50-2]
>     usr/lib/libcap.so
> multilib/lib32-libcap 2.49-1 [installed: 2.50-1]
>     usr/lib32/libcap.so
>
> $ yay -F cap-ng.h
> core/libcap-ng 0.8.2-1 [installed]
>     usr/include/cap-ng.h
>
> $ ls -l /usr/include/cap*
> -rw-r--r-- 1 root root 3402 dez  9  2020 /usr/include/cap-ng.h
>
> $ ls -l /usr/lib/libcap*
> lrwxrwxrwx 1 root root    18 dez  9  2020 /usr/lib/libcap-ng.so ->
> libcap-ng.so.0.0.0
> lrwxrwxrwx 1 root root    18 dez  9  2020 /usr/lib/libcap-ng.so.0 ->
> libcap-ng.so.0.0.0
> -rwxr-xr-x 1 root root 26424 dez  9  2020 /usr/lib/libcap-ng.so.0.0.0
> lrwxrwxrwx 1 root root    11 jun  7 14:25 /usr/lib/libcap.so -> libcap.so.2
> lrwxrwxrwx 1 root root    14 jun  7 14:25 /usr/lib/libcap.so.2 -> libcap.so.2.50
> -rw-r--r-- 1 root root 38704 jun  7 14:25 /usr/lib/libcap.so.2.50
>
> https://archlinux.org/packages/core/x86_64/libcap/
> https://archlinux.org/packages/core/x86_64/libcap-ng/
>
> Anything, please contact me. I want to help.

Apologies I missed adding the list in my previous reply.

I think your distribution is missing static libcap

$ dpkg -L libcap-dev

[...]

/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libcap.a
/usr/lib/x86_64-linux-gnu/libpsx.a

[...]

It seems like arch does not have them:

https://bbs.archlinux.org/viewtopic.php?id=245303

and they don't plan to either. So you can either build the library locally
or possibly move to a distribution that provides static linking.

[incase we decide to use the static linking for vmtest.sh]

Re: kernel bpf test_progs - vm wrong libc version

[Geyslan G. Bem]

On Tue, 15 Jun 2021 at 12:58, KP Singh <kpsingh@kernel.org> wrote:
>
> On Tue, Jun 15, 2021 at 4:57 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> >
> > On Tue, 15 Jun 2021 at 11:33, KP Singh <kpsingh@kernel.org> wrote:
> > >
> > > On Tue, Jun 15, 2021 at 2:34 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> > > >
> > > > On Tue, 15 Jun 2021 at 06:58, KP Singh <kpsingh@kernel.org> wrote:
> > > > >
> > > > > On Tue, Jun 15, 2021 at 10:06 AM Jussi Maki <joamaki@gmail.com> wrote:
> > > > > >
> > > > > > On Tue, Jun 15, 2021 at 8:28 AM Andrii Nakryiko
> > > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > > > It seems kind of silly to update our perfectly working image just
> > > > > > > because a new version of glibc was released. Is there any way for you
> > > > > > > to down-grade glibc or build it in some compatibility mode, etc?
> > > > > > > selftests don't really rely on any bleeding-edge features of glibc.
> > > > > >
> > > > > > I've also hit this issue as Ubuntu 21.04 ships with glibc 2.33. I
> > > > > > ended up solving it the hard way by rebuilding the image (I needed few
> > > > > > other tools at the time anyway). Definitely agree it's a bit silly if
> > > > > > we'd need to bump the image every time there's a new glibc version out
> > > > > > there. I did try and see if there's a way to build against newer
> > > > > > glibc, but target older versions and I didn't find a way to do that.
> > > > > > Would statically linking test-progs be an option to avoid this kind of
> > > > > > breakage in the future?
> > > > >
> > > > > I think static linking tests_progs is the only real way one can solve this.
> > > > > Even if we keep updating the image, there will still be users that will hit
> > > > > glibc version issues.
> > > >
> > > > I agree once the image remains static.
> > > >
> > > > >
> > > > > Andrii, Maybe we can have a mode for vmtest.sh can build test_progs
> > > > > statically?
> > > > >
> > > > > maybe something like:
> > > >
> > > > These changes generates the output:
> > > >
> > > >   BINARY   test_maps
> > > > /usr/bin/ld: cannot find -lcap
> > > > collect2: error: ld returned 1 exit status
> > > > make: *** [Makefile:492:
> > > > /home/uzu/code/bpf-next/tools/testing/selftests/bpf/test_maps] Error 1
> > > >
> > > > libcap and acl are installed
> > >
> > > Are you sure you have libcap-dev installed? I don't see this on my system.
> >
> > As Arch packages maintain headers, I suppose libcap has everything.
> >
> > $ yay -F libcap.so
> > core/libcap 2.49-1 [installed: 2.50-2]
> >     usr/lib/libcap.so
> > multilib/lib32-libcap 2.49-1 [installed: 2.50-1]
> >     usr/lib32/libcap.so
> >
> > $ yay -F cap-ng.h
> > core/libcap-ng 0.8.2-1 [installed]
> >     usr/include/cap-ng.h
> >
> > $ ls -l /usr/include/cap*
> > -rw-r--r-- 1 root root 3402 dez  9  2020 /usr/include/cap-ng.h
> >
> > $ ls -l /usr/lib/libcap*
> > lrwxrwxrwx 1 root root    18 dez  9  2020 /usr/lib/libcap-ng.so ->
> > libcap-ng.so.0.0.0
> > lrwxrwxrwx 1 root root    18 dez  9  2020 /usr/lib/libcap-ng.so.0 ->
> > libcap-ng.so.0.0.0
> > -rwxr-xr-x 1 root root 26424 dez  9  2020 /usr/lib/libcap-ng.so.0.0.0
> > lrwxrwxrwx 1 root root    11 jun  7 14:25 /usr/lib/libcap.so -> libcap.so.2
> > lrwxrwxrwx 1 root root    14 jun  7 14:25 /usr/lib/libcap.so.2 -> libcap.so.2.50
> > -rw-r--r-- 1 root root 38704 jun  7 14:25 /usr/lib/libcap.so.2.50
> >
> > https://archlinux.org/packages/core/x86_64/libcap/
> > https://archlinux.org/packages/core/x86_64/libcap-ng/
> >
> > Anything, please contact me. I want to help.
>
> Apologies I missed adding the list in my previous reply.
>
> I think your distribution is missing static libcap
>
> $ dpkg -L libcap-dev
>
> [...]
>
> /usr/lib/x86_64-linux-gnu
> /usr/lib/x86_64-linux-gnu/libcap.a
> /usr/lib/x86_64-linux-gnu/libpsx.a
>
> [...]
>
> It seems like arch does not have them:
>
> https://bbs.archlinux.org/viewtopic.php?id=245303

Indeed.

>
> and they don't plan to either. So you can either build the library locally
> or possibly move to a distribution that provides static linking.

I think this would keep things in different host environments
complicated. I'm more likely to create a proper VM to handle kernel
source and bpf tests, since bpf also demands llvm13 (cutting edge)
which is conflicting with other projects.

>
> [incase we decide to use the static linking for vmtest.sh]

It's still a good decision for environments with readily available
static binaries.

Thanks a million for your attention.

-- 
Regards,

Geyslan G. Bem

Re: kernel bpf test_progs - vm wrong libc version

[Andrii Nakryiko]

On Tue, Jun 15, 2021 at 9:42 AM Geyslan G. Bem <geyslan@gmail.com> wrote:
>
> On Tue, 15 Jun 2021 at 12:58, KP Singh <kpsingh@kernel.org> wrote:
> >
> > On Tue, Jun 15, 2021 at 4:57 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> > >
> > > On Tue, 15 Jun 2021 at 11:33, KP Singh <kpsingh@kernel.org> wrote:
> > > >
> > > > On Tue, Jun 15, 2021 at 2:34 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> > > > >
> > > > > On Tue, 15 Jun 2021 at 06:58, KP Singh <kpsingh@kernel.org> wrote:
> > > > > >
> > > > > > On Tue, Jun 15, 2021 at 10:06 AM Jussi Maki <joamaki@gmail.com> wrote:
> > > > > > >
> > > > > > > On Tue, Jun 15, 2021 at 8:28 AM Andrii Nakryiko
> > > > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > > > > It seems kind of silly to update our perfectly working image just
> > > > > > > > because a new version of glibc was released. Is there any way for you
> > > > > > > > to down-grade glibc or build it in some compatibility mode, etc?
> > > > > > > > selftests don't really rely on any bleeding-edge features of glibc.
> > > > > > >
> > > > > > > I've also hit this issue as Ubuntu 21.04 ships with glibc 2.33. I
> > > > > > > ended up solving it the hard way by rebuilding the image (I needed few
> > > > > > > other tools at the time anyway). Definitely agree it's a bit silly if
> > > > > > > we'd need to bump the image every time there's a new glibc version out
> > > > > > > there. I did try and see if there's a way to build against newer
> > > > > > > glibc, but target older versions and I didn't find a way to do that.
> > > > > > > Would statically linking test-progs be an option to avoid this kind of
> > > > > > > breakage in the future?
> > > > > >
> > > > > > I think static linking tests_progs is the only real way one can solve this.
> > > > > > Even if we keep updating the image, there will still be users that will hit
> > > > > > glibc version issues.
> > > > >
> > > > > I agree once the image remains static.
> > > > >
> > > > > >
> > > > > > Andrii, Maybe we can have a mode for vmtest.sh can build test_progs
> > > > > > statically?
> > > > > >
> > > > > > maybe something like:
> > > > >
> > > > > These changes generates the output:
> > > > >
> > > > >   BINARY   test_maps
> > > > > /usr/bin/ld: cannot find -lcap
> > > > > collect2: error: ld returned 1 exit status
> > > > > make: *** [Makefile:492:
> > > > > /home/uzu/code/bpf-next/tools/testing/selftests/bpf/test_maps] Error 1
> > > > >
> > > > > libcap and acl are installed
> > > >
> > > > Are you sure you have libcap-dev installed? I don't see this on my system.
> > >
> > > As Arch packages maintain headers, I suppose libcap has everything.
> > >
> > > $ yay -F libcap.so
> > > core/libcap 2.49-1 [installed: 2.50-2]
> > >     usr/lib/libcap.so
> > > multilib/lib32-libcap 2.49-1 [installed: 2.50-1]
> > >     usr/lib32/libcap.so
> > >
> > > $ yay -F cap-ng.h
> > > core/libcap-ng 0.8.2-1 [installed]
> > >     usr/include/cap-ng.h
> > >
> > > $ ls -l /usr/include/cap*
> > > -rw-r--r-- 1 root root 3402 dez  9  2020 /usr/include/cap-ng.h
> > >
> > > $ ls -l /usr/lib/libcap*
> > > lrwxrwxrwx 1 root root    18 dez  9  2020 /usr/lib/libcap-ng.so ->
> > > libcap-ng.so.0.0.0
> > > lrwxrwxrwx 1 root root    18 dez  9  2020 /usr/lib/libcap-ng.so.0 ->
> > > libcap-ng.so.0.0.0
> > > -rwxr-xr-x 1 root root 26424 dez  9  2020 /usr/lib/libcap-ng.so.0.0.0
> > > lrwxrwxrwx 1 root root    11 jun  7 14:25 /usr/lib/libcap.so -> libcap.so.2
> > > lrwxrwxrwx 1 root root    14 jun  7 14:25 /usr/lib/libcap.so.2 -> libcap.so.2.50
> > > -rw-r--r-- 1 root root 38704 jun  7 14:25 /usr/lib/libcap.so.2.50
> > >
> > > https://archlinux.org/packages/core/x86_64/libcap/
> > > https://archlinux.org/packages/core/x86_64/libcap-ng/
> > >
> > > Anything, please contact me. I want to help.
> >
> > Apologies I missed adding the list in my previous reply.
> >
> > I think your distribution is missing static libcap
> >
> > $ dpkg -L libcap-dev
> >
> > [...]
> >
> > /usr/lib/x86_64-linux-gnu
> > /usr/lib/x86_64-linux-gnu/libcap.a
> > /usr/lib/x86_64-linux-gnu/libpsx.a
> >
> > [...]
> >
> > It seems like arch does not have them:
> >
> > https://bbs.archlinux.org/viewtopic.php?id=245303
>
> Indeed.
>
> >
> > and they don't plan to either. So you can either build the library locally
> > or possibly move to a distribution that provides static linking.
>
> I think this would keep things in different host environments
> complicated. I'm more likely to create a proper VM to handle kernel
> source and bpf tests, since bpf also demands llvm13 (cutting edge)
> which is conflicting with other projects.
>

KP, how do you feel about teaching vmtest.sh to (optionally, if
requested or if we detect that environment clang is too old) checkout
clang and build it before building selftests? So many people would be
grateful for this, I imagine! ;)

> >
> > [incase we decide to use the static linking for vmtest.sh]
>
> It's still a good decision for environments with readily available
> static binaries.

yeah, it's a good option to have at the very least

>
> Thanks a million for your attention.
>
> --
> Regards,
>
> Geyslan G. Bem

Re: kernel bpf test_progs - vm wrong libc version

[KP Singh]

On Tue, Jun 15, 2021 at 9:00 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Jun 15, 2021 at 9:42 AM Geyslan G. Bem <geyslan@gmail.com> wrote:
> >
> > On Tue, 15 Jun 2021 at 12:58, KP Singh <kpsingh@kernel.org> wrote:
> > >
> > > On Tue, Jun 15, 2021 at 4:57 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> > > >
> > > > On Tue, 15 Jun 2021 at 11:33, KP Singh <kpsingh@kernel.org> wrote:
> > > > >
> > > > > On Tue, Jun 15, 2021 at 2:34 PM Geyslan G. Bem <geyslan@gmail.com> wrote:

[...]

> > >
> > > [...]
> > >
> > > It seems like arch does not have them:
> > >
> > > https://bbs.archlinux.org/viewtopic.php?id=245303
> >
> > Indeed.
> >
> > >
> > > and they don't plan to either. So you can either build the library locally
> > > or possibly move to a distribution that provides static linking.
> >
> > I think this would keep things in different host environments
> > complicated. I'm more likely to create a proper VM to handle kernel
> > source and bpf tests, since bpf also demands llvm13 (cutting edge)
> > which is conflicting with other projects.
> >
>
> KP, how do you feel about teaching vmtest.sh to (optionally, if
> requested or if we detect that environment clang is too old) checkout
> clang and build it before building selftests? So many people would be
> grateful for this, I imagine! ;)

I agree, I also want to do it for pahole. It will save a lot of time when a
build error could simply be solved by updating clang and pahole.

>
> > >
> > > [incase we decide to use the static linking for vmtest.sh]
> >
> > It's still a good decision for environments with readily available
> > static binaries.
>
> yeah, it's a good option to have at the very least

Cool, will send a patch for this.

- KP

>
> >
> > Thanks a million for your attention.
> >
> > --
> > Regards,
> >
> > Geyslan G. Bem

Re: kernel bpf test_progs - vm wrong libc version

[Geyslan G. Bem]

On Tue, 15 Jun 2021 at 03:27, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
>
> On Fri, Jun 11, 2021 at 1:23 PM Geyslan G. Bem <geyslan@gmail.com> wrote:
> >
> > Trying to run vmtest.sh from the bpf-next linux
> > tools/testing/selftests/bpf on Arch Linux raises this error:
> >
> > ./test_progs
> > ./test_progs: /usr/lib/libc.so.6: version `GLIBC_2.33' not found
> > (required by ./test_progs)
> >
> > VM:
> > https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/libbpf-vmtest-rootfs-2020.09.27.tar.zst
> >
> > [root@(none) /]# strings /usr/lib/libc.so.6 | grep '^GLIBC_2.' | tail
> > GLIBC_2.30
> > GLIBC_2.5
> > GLIBC_2.9
> > GLIBC_2.7
> > GLIBC_2.6
> > GLIBC_2.18
> > GLIBC_2.11
> > GLIBC_2.16
> > GLIBC_2.13
> > GLIBC_2.2.6
> >
> > It would be nice to have
> > https://github.com/libbpf/libbpf/blob/master/travis-ci/vmtest/configs/INDEX
> > updated to refer to a new image with GLIBC_2.33.
> >
> > Host settings:
> >
> > $ strings /usr/lib/libc.so.6 | grep GLIBC_2.33
> > GLIBC_2.33
> > GLIBC_2.33
> >
>
> It seems kind of silly to update our perfectly working image just
> because a new version of glibc was released. Is there any way for you
> to down-grade glibc or build it in some compatibility mode, etc?
> selftests don't really rely on any bleeding-edge features of glibc.

Yeah. Continuously updating/regenerating the image would be a bit
silly, indeed. It was just the trigger for a better proposal.

I would rely on a self-updating image rather than a static one.
Probably that would be more operative. I don't know.

>
> > $ uname -a
> > Linux hb 5.12.9-arch1-1 #1 SMP PREEMPT Thu, 03 Jun 2021 11:36:13 +0000
> > x86_64 GNU/Linux
> >
> > $ gcc --version
> > gcc (GCC) 11.1.0
> >
> > $ clang --version
> > clang version 13.0.0 (/home/uzu/.cache/yay/llvm-git/llvm-project
> > ad381e39a52604ba07e1e027e7bdec1c287d9089)
> > Target: x86_64-pc-linux-gnu
> > Thread model: posix
> > InstalledDir: /usr/bin
> >
> > P.S.: This issue was started in
> > https://github.com/libbpf/libbpf/issues/321 and brought to here.
> >
> > Thank you.
> >
> > Regards,
> >
> > Geyslan G. Bem



-- 
Regards,

Geyslan G. Bem