[PATCH lttng-tools 1/2] Fix: Libtool fails to find dependent libraries when cross-compiling lttng-tools

[PATCH lttng-tools 1/2] Fix: Libtool fails to find dependent libraries when cross-compiling lttng-tools

[Christian Babeux]

This problem arise when cross compiling and linking libraries with
indirect libraries dependencies (such as liblttng-ust). This "bug" is
caused by an upstream modification in the libtool package on Debian
system. The libtool "link_all_deplibs" flag is set to "no" by default
on linux targets (AFAIK, other distros set it to "unknown").

The chosen solution is to detect such cases via the configure script
and automagically patch the libtool.m4 by forcing the "link_all_deplibs"
to "unknown".

This fixup can be disabled with the appropriate configure flag:

./configure --disable-libtool-linkdep-fixup

Sample configure output on affected systems:

checking for occurence(s) of link_all_deplibs = no in
	 ./config/libtool.m4... 3
configure: WARNING: the detected libtool will not link all
 	   dependencies, forcing link_all_deplibs = unknown

Signed-off-by: Christian Babeux <christian.babeux@efficios.com>
---
 configure.ac | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/configure.ac b/configure.ac
index 17e6b67..3a023cd 100644
--- a/configure.ac
+++ b/configure.ac
@@ -14,6 +14,31 @@ AC_DEFINE_UNQUOTED([VERSION_DESCRIPTION], ["$version_description"], "")
 
 AC_CONFIG_HEADERS([include/config.h])
 
+AC_PROG_GREP
+# libtool link_all_deplibs fixup. See http://bugs.lttng.org/issues/321.
+AC_ARG_ENABLE(libtool-linkdep-fixup,
+	AS_HELP_STRING([--disable-libtool-linkdep-fixup],
+			[disable the libtool fixup for linking all dependent libraries (link_all_deplibs)]),
+	libtool_fixup=$enableval,
+	libtool_fixup=yes)
+
+AS_IF([test "x$libtool_fixup" = "xyes"],
+	    [
+	    libtool_m4="$srcdir/config/libtool.m4"
+	    libtool_flag_pattern=".*link_all_deplibs\s*,\s*\$1\s*)"
+	    AC_MSG_CHECKING([for occurence(s) of link_all_deplibs = no in $libtool_m4])
+	    libtool_flag_pattern_count=$(grep -c "$libtool_flag_pattern\s*=\s*no" $libtool_m4)
+	    AS_IF([test $libtool_flag_pattern_count -ne 0],
+	    [
+		  AC_MSG_RESULT([$libtool_flag_pattern_count])
+		  AC_MSG_WARN([the detected libtool will not link all dependencies, forcing link_all_deplibs = unknown])
+		  sed -i "s/\($libtool_flag_pattern\)\s*=\s*no/\1=unknown/g" $libtool_m4
+		  ],
+		  [
+		  AC_MSG_RESULT([none])
+		  ])
+	    ])
+
 AC_CHECK_HEADERS([ \
 	sys/types.h unistd.h fcntl.h string.h pthread.h limits.h \
 	signal.h stdlib.h sys/un.h sys/socket.h stdlib.h stdio.h \
-- 
1.7.11.3

[PATCH lttng-tools 2/2] Fix: Missing libs dependencies in configure check for lttng-ust-ctl

[Christian Babeux]

The lttng-ust-ctl library depends on liburcu-{common,bp,cds}.
The AC_CHECK_LIBRARY macro can't automatically resolve dependents
libraries (ala libtool), so any additionnals dependencies must
be manually specified.

Also, the AC_CHECK_LIB action-if-found case for the lttng-ust-ctl
check is modified to have a similar behavior as the default action,
which is to define HAVE_LIBxxx and append -lxxx to $LIBS, *except*
for the later step. This is to ensure that any future addition of
AC_CHECK_LIB after the one for lttng-ust-ctl will not need to append
the liburcu dependencies or fail unexpectedly.

Signed-off-by: Christian Babeux <christian.babeux@efficios.com>
---
 configure.ac | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/configure.ac b/configure.ac
index 3a023cd..e4b9eb1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -138,13 +138,16 @@ AC_ARG_ENABLE(lttng-ust,
 	lttng_ust_support=$enableval, lttng_ust_support=yes)
 
 AS_IF([test "x$lttng_ust_support" = "xyes"], [
-	AC_CHECK_LIB([lttng-ust-ctl], [ustctl_create_session], [],
-		[AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])]
+	AC_CHECK_LIB([lttng-ust-ctl], [ustctl_create_session],
+		[
+			AC_DEFINE([HAVE_LIBLTTNG_UST_CTL], [1], [has LTTng-UST control support])
+			lttng_ust_ctl_found=yes
+		],
+		[AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])],
+		[-lurcu-common -lurcu-bp -lurcu-cds]
 	)
 ])
-
-AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [ test "x$ac_cv_lib_lttng_ust_ctl_ustctl_create_session" = "xyes" ])
-
+AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [test "x$lttng_ust_ctl_found" = xyes])
 AC_CHECK_FUNCS([sched_getcpu sysconf])
 
 # check for dlopen
-- 
1.7.11.3

Re: [PATCH lttng-tools 1/2] Fix: Libtool fails to find dependent libraries when cross-compiling lttng-tools

[Mathieu Desnoyers]

* Christian Babeux (christian.babeux@efficios.com) wrote:
> This problem arise when cross compiling and linking libraries with
> indirect libraries dependencies (such as liblttng-ust). This "bug" is
> caused by an upstream modification in the libtool package on Debian
> system. The libtool "link_all_deplibs" flag is set to "no" by default
> on linux targets (AFAIK, other distros set it to "unknown").
> 
> The chosen solution is to detect such cases via the configure script
> and automagically patch the libtool.m4 by forcing the "link_all_deplibs"
> to "unknown".
> 
> This fixup can be disabled with the appropriate configure flag:
> 
> ./configure --disable-libtool-linkdep-fixup
> 
> Sample configure output on affected systems:
> 
> checking for occurence(s) of link_all_deplibs = no in
> 	 ./config/libtool.m4... 3
> configure: WARNING: the detected libtool will not link all
>  	   dependencies, forcing link_all_deplibs = unknown
> 
> Signed-off-by: Christian Babeux <christian.babeux@efficios.com>

Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

> ---
>  configure.ac | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/configure.ac b/configure.ac
> index 17e6b67..3a023cd 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -14,6 +14,31 @@ AC_DEFINE_UNQUOTED([VERSION_DESCRIPTION], ["$version_description"], "")
>  
>  AC_CONFIG_HEADERS([include/config.h])
>  
> +AC_PROG_GREP
> +# libtool link_all_deplibs fixup. See http://bugs.lttng.org/issues/321.
> +AC_ARG_ENABLE(libtool-linkdep-fixup,
> +	AS_HELP_STRING([--disable-libtool-linkdep-fixup],
> +			[disable the libtool fixup for linking all dependent libraries (link_all_deplibs)]),
> +	libtool_fixup=$enableval,
> +	libtool_fixup=yes)
> +
> +AS_IF([test "x$libtool_fixup" = "xyes"],
> +	    [
> +	    libtool_m4="$srcdir/config/libtool.m4"
> +	    libtool_flag_pattern=".*link_all_deplibs\s*,\s*\$1\s*)"
> +	    AC_MSG_CHECKING([for occurence(s) of link_all_deplibs = no in $libtool_m4])
> +	    libtool_flag_pattern_count=$(grep -c "$libtool_flag_pattern\s*=\s*no" $libtool_m4)
> +	    AS_IF([test $libtool_flag_pattern_count -ne 0],
> +	    [
> +		  AC_MSG_RESULT([$libtool_flag_pattern_count])
> +		  AC_MSG_WARN([the detected libtool will not link all dependencies, forcing link_all_deplibs = unknown])
> +		  sed -i "s/\($libtool_flag_pattern\)\s*=\s*no/\1=unknown/g" $libtool_m4
> +		  ],
> +		  [
> +		  AC_MSG_RESULT([none])
> +		  ])
> +	    ])
> +
>  AC_CHECK_HEADERS([ \
>  	sys/types.h unistd.h fcntl.h string.h pthread.h limits.h \
>  	signal.h stdlib.h sys/un.h sys/socket.h stdlib.h stdio.h \
> -- 
> 1.7.11.3
> 
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

Re: [PATCH lttng-tools 2/2] Fix: Missing libs dependencies in configure check for lttng-ust-ctl

[Mathieu Desnoyers]

* Christian Babeux (christian.babeux@efficios.com) wrote:
> The lttng-ust-ctl library depends on liburcu-{common,bp,cds}.
> The AC_CHECK_LIBRARY macro can't automatically resolve dependents
> libraries (ala libtool), so any additionnals dependencies must
> be manually specified.
> 
> Also, the AC_CHECK_LIB action-if-found case for the lttng-ust-ctl
> check is modified to have a similar behavior as the default action,
> which is to define HAVE_LIBxxx and append -lxxx to $LIBS, *except*
> for the later step. This is to ensure that any future addition of
> AC_CHECK_LIB after the one for lttng-ust-ctl will not need to append
> the liburcu dependencies or fail unexpectedly.

not sure I understand why all the changes you are doing are needed. What
happens if you just do:

-             [AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])]
+             [AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])],
+             [-lurcu-common -lurcu-bp -lurcu-cds]

?

> 
> Signed-off-by: Christian Babeux <christian.babeux@efficios.com>
> ---
>  configure.ac | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/configure.ac b/configure.ac
> index 3a023cd..e4b9eb1 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -138,13 +138,16 @@ AC_ARG_ENABLE(lttng-ust,
>  	lttng_ust_support=$enableval, lttng_ust_support=yes)
>  
>  AS_IF([test "x$lttng_ust_support" = "xyes"], [
> -	AC_CHECK_LIB([lttng-ust-ctl], [ustctl_create_session], [],
> -		[AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])]
> +	AC_CHECK_LIB([lttng-ust-ctl], [ustctl_create_session],
> +		[
> +			AC_DEFINE([HAVE_LIBLTTNG_UST_CTL], [1], [has LTTng-UST control support])

wasn't HAVE_LIBLTTNG_UST_CTL already defined by the AM_CONDITIONAL below ?

> +			lttng_ust_ctl_found=yes

why are you adding this ?

> +		],
> +		[AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])],
> +		[-lurcu-common -lurcu-bp -lurcu-cds]
>  	)
>  ])
> -
> -AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [ test "x$ac_cv_lib_lttng_ust_ctl_ustctl_create_session" = "xyes" ])
> -
> +AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [test "x$lttng_ust_ctl_found" = xyes])

why are you changing this line ?

Thanks,

Mathieu


>  AC_CHECK_FUNCS([sched_getcpu sysconf])
>  
>  # check for dlopen
> -- 
> 1.7.11.3
> 
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

Re: [PATCH lttng-tools 0/2] Configure fixes for cross-compilation

[David Goulet]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Merged!

Christian Babeux:
> Hi all,
> 
> Here are a few fixes to the configure.ac script to accomodate 
> cross-compilation of lttng-tools.
> 
> Patch 1:
> 
> Same fix that was merged in lttng-ust to fix libtool.m4 on target 
> where link_all_deplibs = no.
> 
> Patch 2:
> 
> Fix an issue encountered when cross-compiling. The configure
> script simply fails saying it cannot find the lttng-ust-ctl library
> even thought the proper "-L/path/to/ust" is specified in $LDFLAGS. 
> The AC_CHECK_LIB test fails because it can't find the libraries
> that liblttng-ust-ctl depends on. The fix is to specify the
> dependents libraries (liburcu-{common,bp,cds}) in the
> other-lib-case of AC_CHECK_LIB.
> 
> Thanks,
> 
> Christian
> 
> Christian Babeux (2): Fix: Libtool fails to find dependent
> libraries when cross-compiling lttng-tools Fix: Missing libs
> dependencies in configure check for lttng-ust-ctl
> 
> configure.ac | 38 +++++++++++++++++++++++++++++++++----- 1 file
> changed, 33 insertions(+), 5 deletions(-)
> 
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJQI8K9AAoJEELoaioR9I02GYoIALy1oupw5BbMgfNV5i/gyPrf
3ywUW+s0/Ltl+H4L6hOUq4daUTyUpt+Tla9bBNiP+HXkRX7rxjMnePjN2lk8QJR6
4lW1fWD7IlUNwQkVzKNZh+fv3xu4QyIiWd9zaEmMWIFwibaAcV8yzaI4ZkdrkHXX
r+b2nVeGScx2kamwj5B/CWXr+FLovsstGEdKUy2KmJZ7A9eLg4G4Eyn8R3U7Sn5m
X3t04liPawiZ6g8NTVQiw+O2vBs6ngdklj5DTQ1v4vLb/a2IwPPEnHjrznV1Np6A
T1ONQex+063MYdxp6a0948pXx2X1jBEgeESphoew/Go15LQqgkUBFZHTye3kKdI=
=GgYi
-----END PGP SIGNATURE-----

Re: [PATCH lttng-tools 2/2] Fix: Missing libs dependencies in configure check for lttng-ust-ctl

[Christian Babeux]

Hi Mathieu,

> not sure I understand why all the changes you are doing are needed. What
> happens if you just do:
>
> -             [AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])]
> +             [AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])],
> +             [-lurcu-common -lurcu-bp -lurcu-cds]
>
> ?

If we only change the other-lib case of the AC_CHECK_LIB, subsequent
calls to this macro will fail unexpectedly. For example, the next call
to this macro is the check for dlopen in the dl lib. Since the previous
call (the check for lttng-ust-ctl) succeeded, -llttng-ust-ctl has been
prepended to the $LIBS variable.

The problem arise when trying to link a test program
(the actual test of AC_CHECK_LIB) to check for the dl lib. The linking
will fail for the same reason, because the dependencies to
urcu-{common,bp,cds} have not been manually specified and lttng-ust-ctl
has been added to the $LIBS flags passed to the linker. Hence,
the test for the dl lib will fail, not because it can't find the dl lib,
but because the linking fail as a side effect of adding the lttng-ust-ctl lib.

There is possibly three other ways to fix this issue that I know of:

- The check for lttng-ust-ctl lib should be the last AC_CHECK_LIB.
  This is exactly the situation I was trying to avoid. Someone could
  inadvertently add one after and silently break cross-compilation
  builds.

- Add AC_CHECK_LIB checks for the dependents URCU libs.
  This will add a -lurcu-{common,bp,cds} when linking.
  I'm not fully aware of the repercussions this could have.

- Use PKG_CONFIG support in configure.ac.
  This might be a more elegant and long-term solution so we don't have
  to rely on these kind of trickery to check for libraries
  and correctly support cross-compilation builds.


>> +     AC_CHECK_LIB([lttng-ust-ctl], [ustctl_create_session],
>> +             [
>> +                     AC_DEFINE([HAVE_LIBLTTNG_UST_CTL], [1], [has LTTng-UST control support])
>
> wasn't HAVE_LIBLTTNG_UST_CTL already defined by the AM_CONDITIONAL below ?
>
>> +                     lttng_ust_ctl_found=yes
>
> why are you adding this ?
>

The default behavior of the action-if-found in AC_CHECK_LIB is to
prepend '-llibrary' and defining 'HAVE_LIBlibrary' [1].
Since we don't want to prepend -llttng-ust-ctl, we must manually define
HAVE_LIBlibrary.

The AM_CONDITIONAL does not define the HAVE_LIBLTTNG_UST_CTL
(as in the preprocessor sense) it is the AC_CHECK_LIB check
that normally does (via an underlying call to AC_DEFINE).
AM_CONDITIONAL only make the HAVE_LIBLTTNG_UST_CTL variable
available to use in Makefile.am [2].

>> -
>> -AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [ test "x$ac_cv_lib_lttng_ust_ctl_ustctl_create_session" = "xyes" ])
>> -
>> +AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [test "x$lttng_ust_ctl_found" = xyes])
>
> why are you changing this line ?
>

I only changed the test flag to lttng_ust_ctl_found for clarity sake.
It might not be obvious to everyone that the
$ac_cv_lib_lttng_ust_ctl_ustctl_create_session variable is a cached
test result by autoconf.

I think this patch is not the most obvious. I would like to know
if anyone has any other ideas to resolve this issue.

Thoughts?

Thanks,

Christian

1 - https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Libraries.html#Libraries
2 - https://www.gnu.org/software/automake/manual/html_node/Usage-of-Conditionals.html

Re: [PATCH lttng-tools 2/2] Fix: Missing libs dependencies in configure check for lttng-ust-ctl

[Mathieu Desnoyers]

* Christian Babeux (christian.babeux@efficios.com) wrote:
> Hi Mathieu,
> 
> > not sure I understand why all the changes you are doing are needed. What
> > happens if you just do:
> >
> > -             [AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])]
> > +             [AC_MSG_ERROR([Cannot find LTTng-UST. Use [LDFLAGS]=-Ldir to specify its location, or specify --disable-lttng-ust to build lttng-tools without LTTng-UST support.])],
> > +             [-lurcu-common -lurcu-bp -lurcu-cds]
> >
> > ?
> 
> If we only change the other-lib case of the AC_CHECK_LIB, subsequent
> calls to this macro will fail unexpectedly. For example, the next call
> to this macro is the check for dlopen in the dl lib. Since the previous
> call (the check for lttng-ust-ctl) succeeded, -llttng-ust-ctl has been
> prepended to the $LIBS variable.
> 
> The problem arise when trying to link a test program
> (the actual test of AC_CHECK_LIB) to check for the dl lib. The linking
> will fail for the same reason, because the dependencies to
> urcu-{common,bp,cds} have not been manually specified and lttng-ust-ctl
> has been added to the $LIBS flags passed to the linker. Hence,
> the test for the dl lib will fail, not because it can't find the dl lib,
> but because the linking fail as a side effect of adding the lttng-ust-ctl lib.
> 
> There is possibly three other ways to fix this issue that I know of:
> 
> - The check for lttng-ust-ctl lib should be the last AC_CHECK_LIB.
>   This is exactly the situation I was trying to avoid. Someone could
>   inadvertently add one after and silently break cross-compilation
>   builds.
> 
> - Add AC_CHECK_LIB checks for the dependents URCU libs.
>   This will add a -lurcu-{common,bp,cds} when linking.
>   I'm not fully aware of the repercussions this could have.
> 
> - Use PKG_CONFIG support in configure.ac.
>   This might be a more elegant and long-term solution so we don't have
>   to rely on these kind of trickery to check for libraries
>   and correctly support cross-compilation builds.
> 
> 
> >> +     AC_CHECK_LIB([lttng-ust-ctl], [ustctl_create_session],
> >> +             [
> >> +                     AC_DEFINE([HAVE_LIBLTTNG_UST_CTL], [1], [has LTTng-UST control support])
> >
> > wasn't HAVE_LIBLTTNG_UST_CTL already defined by the AM_CONDITIONAL below ?
> >
> >> +                     lttng_ust_ctl_found=yes
> >
> > why are you adding this ?
> >
> 
> The default behavior of the action-if-found in AC_CHECK_LIB is to
> prepend '-llibrary' and defining 'HAVE_LIBlibrary' [1].
> Since we don't want to prepend -llttng-ust-ctl, we must manually define
> HAVE_LIBlibrary.
> 
> The AM_CONDITIONAL does not define the HAVE_LIBLTTNG_UST_CTL
> (as in the preprocessor sense) it is the AC_CHECK_LIB check
> that normally does (via an underlying call to AC_DEFINE).
> AM_CONDITIONAL only make the HAVE_LIBLTTNG_UST_CTL variable
> available to use in Makefile.am [2].
> 
> >> -
> >> -AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [ test "x$ac_cv_lib_lttng_ust_ctl_ustctl_create_session" = "xyes" ])
> >> -
> >> +AM_CONDITIONAL([HAVE_LIBLTTNG_UST_CTL], [test "x$lttng_ust_ctl_found" = xyes])
> >
> > why are you changing this line ?
> >
> 
> I only changed the test flag to lttng_ust_ctl_found for clarity sake.
> It might not be obvious to everyone that the
> $ac_cv_lib_lttng_ust_ctl_ustctl_create_session variable is a cached
> test result by autoconf.
> 
> I think this patch is not the most obvious. I would like to know
> if anyone has any other ideas to resolve this issue.

ok, this explanation is fine. Now I understand that by specifying
anything other than [] in the AC_CHECK_LIB, you override the default
behavior, and hence you don't have the LIBS+=.

Good!

Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

> 
> Thoughts?
> 
> Thanks,
> 
> Christian
> 
> 1 - https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Libraries.html#Libraries
> 2 - https://www.gnu.org/software/automake/manual/html_node/Usage-of-Conditionals.html

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com