[Buildroot] [PATCH 1/2] toolchain: support mismatched merged usr

[Buildroot] [PATCH 1/2] toolchain: support mismatched merged usr

[Norbert Lange]

Look at the case where the source toolchain has non-merged usr,
yet the target will have merged usr.

sysroot/lib/ld-musl-x86_64.so.1 -> ../usr/lib/libc.so
sysroot/usr/lib/libc.so

What happens is that buildroot copies the ld-*so* symlink
into usr/lib, at which point it becomes broken.

We now detect these broken symlinks, then try to find the target
binary in the library directories and fix the link.

Fix the case where the lib directory is a symlink, and no ld-*so*
is installed by adding -H to find.

Also use `cp -t` instead of some rarely used xargs tricks.

Signed-off-by: Norbert Lange <nolange79@gmail.com>
---
 toolchain/helpers.mk | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index ef8e9a5f64..aaf2aecd80 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -135,8 +135,17 @@ copy_toolchain_sysroot = \
 			$(call simplify_symlink,$$i,$(STAGING_DIR)) ; \
 		done ; \
 	fi ; \
-	if [[ ! $$(find $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
-		find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 -I % cp % $(STAGING_DIR)/lib/; \
+	for i in $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -xtype l); do \
+		LINKTARGET=`readlink $$i`; \
+		rm $$i; \
+		NEWLINKTARGET=$$(find -H $(STAGING_DIR)/$${ARCH_LIB_DIR} $(STAGING_DIR)/lib $(STAGING_DIR)/usr/$${ARCH_LIB_DIR} $(STAGING_DIR)/usr/lib -name "`basename $${LINKTARGET}`" -print -quit); \
+		if [ -n "$${NEWLINKTARGET}" -a -e "$${NEWLINKTARGET}" ]; then \
+			ln -sr $${NEWLINKTARGET} $$i; \
+			echo "Symlinking $$i -> `readlink $$i`" ; \
+		fi; \
+	done; \
+	if [[ ! $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
+		find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 cp -t $(STAGING_DIR)/lib/; \
 	fi ; \
 	if [ `readlink -f $${SYSROOT_DIR}` != `readlink -f $${ARCH_SYSROOT_DIR}` ] ; then \
 		if [ ! -d $${ARCH_SYSROOT_DIR}/usr/include ] ; then \
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2] toolchain: prevent infinite loop in copy_toolchain_lib_root

[Norbert Lange]

This can happen when there are broken symlinks.

Signed-off-by: Norbert Lange <nolange79@gmail.com>
---
 toolchain/helpers.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index aaf2aecd80..7623d68123 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -20,6 +20,7 @@ copy_toolchain_lib_root = \
 			if test -h $${LIBPATH} ; then \
 				cp -d $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
 				LIBPATH="`readlink -f $${LIBPATH}`"; \
+				[ -n "$${LIBPATH}" ] || break; \
 			elif test -f $${LIBPATH}; then \
 				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
 				break ; \
-- 
2.34.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] toolchain: support mismatched merged usr

[Arnout Vandecappelle]

On 15/02/2022 13:46, Norbert Lange wrote:
> Look at the case where the source toolchain has non-merged usr,
> yet the target will have merged usr.
> 
> sysroot/lib/ld-musl-x86_64.so.1 -> ../usr/lib/libc.so
> sysroot/usr/lib/libc.so

  Where do you get an external toolchain with such a weird layout? Normally libc 
is in sysroot/lib and ld-musl*.so.1 is a symlink to either /lib/libc.so or 
../lib/libc.so.


  Regards,
  Arnout

> 
> What happens is that buildroot copies the ld-*so* symlink
> into usr/lib, at which point it becomes broken.
> 
> We now detect these broken symlinks, then try to find the target
> binary in the library directories and fix the link.
> 
> Fix the case where the lib directory is a symlink, and no ld-*so*
> is installed by adding -H to find.
> 
> Also use `cp -t` instead of some rarely used xargs tricks.
> 
> Signed-off-by: Norbert Lange <nolange79@gmail.com>
> ---
>   toolchain/helpers.mk | 13 +++++++++++--
>   1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> index ef8e9a5f64..aaf2aecd80 100644
> --- a/toolchain/helpers.mk
> +++ b/toolchain/helpers.mk
> @@ -135,8 +135,17 @@ copy_toolchain_sysroot = \
>   			$(call simplify_symlink,$$i,$(STAGING_DIR)) ; \
>   		done ; \
>   	fi ; \
> -	if [[ ! $$(find $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
> -		find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 -I % cp % $(STAGING_DIR)/lib/; \
> +	for i in $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -xtype l); do \
> +		LINKTARGET=`readlink $$i`; \
> +		rm $$i; \
> +		NEWLINKTARGET=$$(find -H $(STAGING_DIR)/$${ARCH_LIB_DIR} $(STAGING_DIR)/lib $(STAGING_DIR)/usr/$${ARCH_LIB_DIR} $(STAGING_DIR)/usr/lib -name "`basename $${LINKTARGET}`" -print -quit); \
> +		if [ -n "$${NEWLINKTARGET}" -a -e "$${NEWLINKTARGET}" ]; then \
> +			ln -sr $${NEWLINKTARGET} $$i; \
> +			echo "Symlinking $$i -> `readlink $$i`" ; \
> +		fi; \
> +	done; \
> +	if [[ ! $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
> +		find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 cp -t $(STAGING_DIR)/lib/; \
>   	fi ; \
>   	if [ `readlink -f $${SYSROOT_DIR}` != `readlink -f $${ARCH_SYSROOT_DIR}` ] ; then \
>   		if [ ! -d $${ARCH_SYSROOT_DIR}/usr/include ] ; then \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] toolchain: support mismatched merged usr

[Norbert Lange]

Am Sa., 5. März 2022 um 16:09 Uhr schrieb Arnout Vandecappelle <arnout@mind.be>:
>
>
>
> On 15/02/2022 13:46, Norbert Lange wrote:
> > Look at the case where the source toolchain has non-merged usr,
> > yet the target will have merged usr.
> >
> > sysroot/lib/ld-musl-x86_64.so.1 -> ../usr/lib/libc.so
> > sysroot/usr/lib/libc.so
>
>   Where do you get an external toolchain with such a weird layout? Normally libc
> is in sysroot/lib and ld-musl*.so.1 is a symlink to either /lib/libc.so or
> ../lib/libc.so.
>
>
>   Regards,
>   Arnout
>

the toolchain is built with crosstool-ng

> >
> > What happens is that buildroot copies the ld-*so* symlink
> > into usr/lib, at which point it becomes broken.
> >
> > We now detect these broken symlinks, then try to find the target
> > binary in the library directories and fix the link.
> >
> > Fix the case where the lib directory is a symlink, and no ld-*so*
> > is installed by adding -H to find.
> >
> > Also use `cp -t` instead of some rarely used xargs tricks.
> >
> > Signed-off-by: Norbert Lange <nolange79@gmail.com>
> > ---
> >   toolchain/helpers.mk | 13 +++++++++++--
> >   1 file changed, 11 insertions(+), 2 deletions(-)
> >
> > diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> > index ef8e9a5f64..aaf2aecd80 100644
> > --- a/toolchain/helpers.mk
> > +++ b/toolchain/helpers.mk
> > @@ -135,8 +135,17 @@ copy_toolchain_sysroot = \
> >                       $(call simplify_symlink,$$i,$(STAGING_DIR)) ; \
> >               done ; \
> >       fi ; \
> > -     if [[ ! $$(find $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
> > -             find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 -I % cp % $(STAGING_DIR)/lib/; \
> > +     for i in $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -xtype l); do \
> > +             LINKTARGET=`readlink $$i`; \
> > +             rm $$i; \
> > +             NEWLINKTARGET=$$(find -H $(STAGING_DIR)/$${ARCH_LIB_DIR} $(STAGING_DIR)/lib $(STAGING_DIR)/usr/$${ARCH_LIB_DIR} $(STAGING_DIR)/usr/lib -name "`basename $${LINKTARGET}`" -print -quit); \
> > +             if [ -n "$${NEWLINKTARGET}" -a -e "$${NEWLINKTARGET}" ]; then \
> > +                     ln -sr $${NEWLINKTARGET} $$i; \
> > +                     echo "Symlinking $$i -> `readlink $$i`" ; \
> > +             fi; \
> > +     done; \
> > +     if [[ ! $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
> > +             find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 cp -t $(STAGING_DIR)/lib/; \
> >       fi ; \
> >       if [ `readlink -f $${SYSROOT_DIR}` != `readlink -f $${ARCH_SYSROOT_DIR}` ] ; then \
> >               if [ ! -d $${ARCH_SYSROOT_DIR}/usr/include ] ; then \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/2] toolchain: support mismatched merged usr

[Arnout Vandecappelle]

  Hi Norbert,

  Sorry that this has been laying around so long without reaction. The thing is 
that this external toolchain sysroot stuff is incredibly fragile and even worse, 
difficult to understand. So there isn't much motivation to accept patches. We 
probably should have some tests for it...

On 15/02/2022 13:46, Norbert Lange wrote:
> Look at the case where the source toolchain has non-merged usr,
> yet the target will have merged usr.
> 
> sysroot/lib/ld-musl-x86_64.so.1 -> ../usr/lib/libc.so
> sysroot/usr/lib/libc.so
> 
> What happens is that buildroot copies the ld-*so* symlink
> into usr/lib, at which point it becomes broken.
> 
> We now detect these broken symlinks, then try to find the target
> binary in the library directories and fix the link.
> 
> Fix the case where the lib directory is a symlink, and no ld-*so*
> is installed by adding -H to find.

  I don't understand what's the issue there. $(STAGING_DIR)/lib is always a 
symlink in merged usr, and never a symlink in non-merged usr. Also, the -H 
parameter has no effect on the paths supplied in the command line (i.e. 
$(STAGING_DIR)/lib itself), only on the symlinks within it. So I don't 
understand what you're trying to fix here. Maybe you have a tuple->. symlink 
inside the lib directory? But even that won't be followed by find because it 
sees that it's a directory it already encountered. Or maybe it's a 
tuple->../usr/lib/tuple symlink? That also won't be followed by find because 
it's a broken symlink.

  So, can you explain better (in the commit message) in which situation this is 
needed?

> Also use `cp -t` instead of some rarely used xargs tricks.

  That's pretty much in the eye of the beholder. I regularly use xargs -I, I 
never used cp -t. We use neither of these constructs anywhere else in Buildroot. 
So if anything, it should probably be changed into find -exec which we do use 
elsewhere (though I honestly find it much more annoying to use than xargs -I).

  Anyway, such an unrelated change should be a separate patch, so we can easily 
skip it if we don't agree (or apply it if we don't agree with the rest).

> 
> Signed-off-by: Norbert Lange <nolange79@gmail.com>
> ---
>   toolchain/helpers.mk | 13 +++++++++++--
>   1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> index ef8e9a5f64..aaf2aecd80 100644
> --- a/toolchain/helpers.mk
> +++ b/toolchain/helpers.mk
> @@ -135,8 +135,17 @@ copy_toolchain_sysroot = \

  There's a huge comment above copy_toolchain_sysroot that explains in detail 
what it does. It should really be interspersed with the code itself, but due to 
make limitations that's not possible. Anyway, that comment should be extended 
with an explanation of the why and the how of the new block.

>   			$(call simplify_symlink,$$i,$(STAGING_DIR)) ; \
>   		done ; \
>   	fi ; \
> -	if [[ ! $$(find $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
> -		find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 -I % cp % $(STAGING_DIR)/lib/; \
> +	for i in $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -xtype l); do \

  Should there ever be more than one? IOW shouldn't there be a -quit?

  Also, we usually use backticks instead of $$()

> +		LINKTARGET=`readlink $$i`; \
> +		rm $$i; \
> +		NEWLINKTARGET=$$(find -H $(STAGING_DIR)/$${ARCH_LIB_DIR} $(STAGING_DIR)/lib $(STAGING_DIR)/usr/$${ARCH_LIB_DIR} $(STAGING_DIR)/usr/lib -name "`basename $${LINKTARGET}`" -print -quit); \

  What you actually want here is to take the first one that matches, right? 
Also, it should always be in one of those directories themselves, and not in one 
of the subdirectories I think? In that case, I think it's more clear if it's a 
loop rather than find. So something like:

	for libdir in $(STAGING_DIR)/$${ARCH_LIB_DIR} $(STAGING_DIR)/lib 
$(STAGING_DIR)/usr/$${ARCH_LIB_DIR} $(STAGING_DIR)/usr/lib; do \
		NEWLINKTARGET="$${libdir}/`basename $${LINKTARGET}`"; \
		if [ -n "$${NEWLINKTARGET}" -a -e "$${NEWLINKTARGET}" ]; then \
			echo "Symlinking $$i -> `readlink $$i`" ; \
			ln -sr $${NEWLINKTARGET} $$i; \
			break;
		fi; \
	done; \

(this was just some quick coding, I probably missed a lot of things).

> +		if [ -n "$${NEWLINKTARGET}" -a -e "$${NEWLINKTARGET}" ]; then \

  If this is not true, was it a good idea to delete the symlink? Or maybe we 
should even error out in that case?

> +			ln -sr $${NEWLINKTARGET} $$i; \
> +			echo "Symlinking $$i -> `readlink $$i`" ; \
> +		fi; \
> +	done; \
> +	if [[ ! $$(find -H $(STAGING_DIR)/lib -name 'ld*.so.*' -print -quit) ]]; then \
> +		find $${ARCH_SYSROOT_DIR}/lib -name 'ld*.so.*' -print0 | xargs -0 cp -t $(STAGING_DIR)/lib/; \

  I wonder if this bit couldn't be merged with the above (skipping the -xtype in 
the find of course).


  Regards,
  Arnout

>   	fi ; \
>   	if [ `readlink -f $${SYSROOT_DIR}` != `readlink -f $${ARCH_SYSROOT_DIR}` ] ; then \
>   		if [ ! -d $${ARCH_SYSROOT_DIR}/usr/include ] ; then \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/2] toolchain: prevent infinite loop in copy_toolchain_lib_root

[Arnout Vandecappelle]

On 15/02/2022 13:46, Norbert Lange wrote:
> This can happen when there are broken symlinks.
> 
> Signed-off-by: Norbert Lange <nolange79@gmail.com>
> ---
>   toolchain/helpers.mk | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> index aaf2aecd80..7623d68123 100644
> --- a/toolchain/helpers.mk
> +++ b/toolchain/helpers.mk
> @@ -20,6 +20,7 @@ copy_toolchain_lib_root = \
>   			if test -h $${LIBPATH} ; then \
>   				cp -d $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
>   				LIBPATH="`readlink -f $${LIBPATH}`"; \
> +				[ -n "$${LIBPATH}" ] || break; \

  I again don't understand how this can happen. LIBPATH exists and is a symlink 
(tested a few lines above), so readlink will always return the contents of that 
symlink. I don't think that can ever be empty, but with the -f (full path) 
option it can definitely not be empty: readlink doesn't care about broken 
symlinks, and even if it has too many .. components it will just print something 
based on /.

  Can you give an example of such a broken symlink where this can happen?

  Regards,
  Arnout


>   			elif test -f $${LIBPATH}; then \
>   				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
>   				break ; \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot