[Buildroot] [Bug 15181] New: Building libcurl fails due to faulty package definition

[Buildroot] [Bug 15181] New: Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

            Bug ID: 15181
           Summary: Building libcurl fails due to faulty package
                    definition
           Product: buildroot
           Version: 2022.02.8
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned@buildroot.uclibc.org
          Reporter: dmrauh@posteo.de
                CC: buildroot@uclibc.org
  Target Milestone: ---

Hi all,

I'm trying to compile "libcurl" using Buildroot 2022.02.8 with the following
relevant options in my .config:

BR2_PACKAGE_LIBCURL=y
BR2_PACKAGE_LIBCURL_OPENSSL=y
BR2_PACKAGE_OPENSSL=y


Unfortunately the build is reproducibly failing during "libcurl-configure" with
the following symptoms:

...
configure: PKG_CONFIG_LIBDIR will be set to
"/home/rauhdomi/build/out/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/pkgconfig"
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lcrypto "
configure: pkg-config: SSL_LDFLAGS: "-L/usr/lib64 "
configure: pkg-config: SSL_CPPFLAGS: "-I/usr/include "
checking for HMAC_Update in -lcrypto... no
checking for HMAC_Init_ex in -lcrypto... no
checking OpenSSL linking with -ldl... no
checking OpenSSL linking with -ldl and -lpthread... no
configure: OPT_OPENSSL:
/home/rauhdomi/build/out/host/x86_64-buildroot-linux-gnu/sysroot/usr
configure: OPENSSL_ENABLED: 
configure: error: --with-openssl was given but OpenSSL could not be detected
make[1]: *** [package/pkg-generic.mk:283:
/home/rauhdomi/build/out/build/libcurl-7.84.0/.stamp_configured] Error 1
make: *** [Makefile:40: _all] Error 2

After poking around for a bit I narrowed it down to the following part of
libcurl's package definition in Buildroot "libcurl.mk":

# configure adds the cross openssl dir to LD_LIBRARY_PATH which screws up
# native stuff during the rest of configure when target == host.
# Fix it by setting LD_LIBRARY_PATH to something sensible so those libs
# are found first.
LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if
$(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)/lib:/usr/lib

Since on my build system "pkgconf" is also installed through my distro's
package manager (in version 1.8.0), its "libpkgconf.so.3" is used instead of
the one part of "$(HOST_DIR)/lib", when building "libcurl". Apparently this has
never been a problem before because the versions were semantically compatible,
which isn't the case any longer.

I guess this bug has the potential to affect any user having "pkgconf" already
installed on their system.

My proposal on how to fix the bug would be as follows:

diff --git a/package/libcurl/libcurl.mk b/package/libcurl/libcurl.mk
index e241bd1c88..f6f5501f83 100644
--- a/package/libcurl/libcurl.mk
+++ b/package/libcurl/libcurl.mk
@@ -46,7 +46,7 @@ LIBCURL_DEPENDENCIES += openssl
 # native stuff during the rest of configure when target == host.
 # Fix it by setting LD_LIBRARY_PATH to something sensible so those libs
 # are found first.
-LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if
$(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)/lib:/usr/lib
+LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if
$(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)$(HOST_DIR)/lib:/lib:/usr/lib
 LIBCURL_CONF_OPTS += --with-ssl=$(STAGING_DIR)/usr \
        --with-ca-path=/etc/ssl/certs
 else

This way we always make sure that libs part of the SDK generated by Buildroot
take precedence over the ones coming from the build system.

Best regards,
Dominik

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [Bug 15181] Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

--- Comment #1 from Thomas Petazzoni <thomas.petazzoni@bootlin.com> ---
Thanks for the great investigation.

One question is whether the LD_LIBRARY_PATH trick is still needed? It was added
back in 2009, so perhaps things have changed in libcurl?

This commit in libcurl:

commit 2d4c2152c9eb3dbdf943de46ed8fc11285f1b90b
Author: Daniel Stenberg <daniel@haxx.se>
Date:   Fri Apr 13 14:07:39 2018 +0200

seems to have reworked how this works, including for openssl.

Also, I have removed the LD_LIBRARY_PATH tricked, and built the following
configuration on an x86-64 machine (to have the target == host case):

BR2_x86_64=y
BR2_x86_corei7=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_X86_64_CORE_I7_GLIBC_BLEEDING_EDGE=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_OPENSSL=y
BR2_PACKAGE_LIBCURL=y
# BR2_TARGET_ROOTFS_TAR is not set

and it built just fine.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [Bug 15181] Building libcurl fails due to faulty package definition

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=15181

--- Comment #2 from Jookia <contact@jookia.org> ---
My friend Xogium hit this issue this month and after a few debugging sessions
we tracked this down. Here's what I've found:

- Arch Linux's host toolchain now sets runpath instead of rpath
- libcurl sets LD_LIBRARY_PATH to /lib:/usr/lib
- Buildroot pkgconf and Arch Linux pkgconf versions line up
- Arch Linux pkgconf library is installed to /usr/lib/libpkgconf.so
- Buildroot pkgconf uses the host libpkgconf.so due to LD_LIBRARY_PATH
- this is respected because rpath does not override it
- The Buildroot pkg-conf script sets LIBPATH to build/host/bin/../sysroot/...
and SYSROOT to build/host/sysroot
- pkgconf 1.8 silently ignores LIBPATH if it's outside sysroot
- pkgconf 1.8 does not normalize paths
- pkgconf reverted LIBPATH to the one in the host's libpkg.so, /lib and
/usr/lib
- curl build tries to find openssl but can't

Two fixes are needed:

- libcurl build should not set LD_LIBRARY_PATH
- the pkg-config wrapper should use normalized paths

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot