All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] staging: Handle races between binaries and their libs
@ 2020-02-14 13:02 Richard Purdie
  2020-02-14 14:46 ` Khem Raj
  2020-02-14 17:01 ` Peter Kjellerstedt
  0 siblings, 2 replies; 6+ messages in thread
From: Richard Purdie @ 2020-02-14 13:02 UTC (permalink / raw)
  To: openembedded-core

There is a long standing issue where a binary could be installed into the
sysroot before its library dependencies. We've always argued nothing should
use the binary until its been installed by a dependency but there are issues
around binaries which conflict with the host system, for example patch,
python3, gzip and more.

With the recent patch changes we've see issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0  Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

which is a symptom of this issue (libattr-native is a dependency of patch-native).

There are other ways to fix this such as disabling libattr in patch, installing
patch to a subdirectory and requiring PATH manipulation and so on.

We can simply fix the staging code to handle /bin/ after everything else so lets
do that and avoid all these other complications.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/staging.bbclass | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 4dd2ed01010..530e23b1853 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -470,6 +470,7 @@ python extend_recipe_sysroot() {
         elif os.path.lexists(depdir + "/" + c):
             os.unlink(depdir + "/" + c)
 
+    binfiles = {}
     # Now handle installs
     for dep in configuredeps:
         c = setscenedeps[dep][0]
@@ -562,7 +563,16 @@ python extend_recipe_sysroot() {
                     if l.endswith("/"):
                         staging_copydir(l, targetdir, dest, seendirs)
                         continue
-                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+                    if "/bin/" in l:
+                        # defer /bin/* files until last in case they need libs
+                        binfiles[l] = (targetdir, dest)
+                    else:
+                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+    # Handle deferred binfiles
+    for l in binfiles:
+        (targetdir, dest) = binfiles[l]
+        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
 
     bb.note("Installed into sysroot: %s" % str(msg_adding))
     bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] staging: Handle races between binaries and their libs
  2020-02-14 13:02 [PATCH] staging: Handle races between binaries and their libs Richard Purdie
@ 2020-02-14 14:46 ` Khem Raj
  2020-02-14 14:50   ` Richard Purdie
  2020-02-14 17:01 ` Peter Kjellerstedt
  1 sibling, 1 reply; 6+ messages in thread
From: Khem Raj @ 2020-02-14 14:46 UTC (permalink / raw)
  To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer

On Fri, Feb 14, 2020 at 5:03 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> There is a long standing issue where a binary could be installed into the
> sysroot before its library dependencies. We've always argued nothing should
> use the binary until its been installed by a dependency but there are issues
> around binaries which conflict with the host system, for example patch,
> python3, gzip and more.
>
> With the recent patch changes we've see issues like:
> ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0  Output:
> Applying patch 0009-Change-order-of-CFLAGS.patch
> patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
> Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)
>
> which is a symptom of this issue (libattr-native is a dependency of patch-native).
>

ah so this was the problem I wondered

> There are other ways to fix this such as disabling libattr in patch, installing
> patch to a subdirectory and requiring PATH manipulation and so on.
>
> We can simply fix the staging code to handle /bin/ after everything else so lets
> do that and avoid all these other complications.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/staging.bbclass | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
> index 4dd2ed01010..530e23b1853 100644
> --- a/meta/classes/staging.bbclass
> +++ b/meta/classes/staging.bbclass
> @@ -470,6 +470,7 @@ python extend_recipe_sysroot() {
>          elif os.path.lexists(depdir + "/" + c):
>              os.unlink(depdir + "/" + c)
>
> +    binfiles = {}
>      # Now handle installs
>      for dep in configuredeps:
>          c = setscenedeps[dep][0]
> @@ -562,7 +563,16 @@ python extend_recipe_sysroot() {
>                      if l.endswith("/"):
>                          staging_copydir(l, targetdir, dest, seendirs)
>                          continue
> -                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
> +                    if "/bin/" in l:
> +                        # defer /bin/* files until last in case they need libs
> +                        binfiles[l] = (targetdir, dest)
> +                    else:
> +                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
> +
> +    # Handle deferred binfiles
> +    for l in binfiles:
> +        (targetdir, dest) = binfiles[l]
> +        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
>
>      bb.note("Installed into sysroot: %s" % str(msg_adding))
>      bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
> --
> 2.20.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] staging: Handle races between binaries and their libs
  2020-02-14 14:46 ` Khem Raj
@ 2020-02-14 14:50   ` Richard Purdie
  2020-02-14 15:44     ` Khem Raj
  0 siblings, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2020-02-14 14:50 UTC (permalink / raw)
  To: Khem Raj; +Cc: Patches and discussions about the oe-core layer

On Fri, 2020-02-14 at 06:46 -0800, Khem Raj wrote:
> On Fri, Feb 14, 2020 at 5:03 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > There is a long standing issue where a binary could be installed
> > into the
> > sysroot before its library dependencies. We've always argued
> > nothing should
> > use the binary until its been installed by a dependency but there
> > are issues
> > around binaries which conflict with the host system, for example
> > patch,
> > python3, gzip and more.
> > 
> > With the recent patch changes we've see issues like:
> > ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error:
> > 'quilt --quiltrc /home/pokybuild/yocto-
> > worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-
> > linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-
> > native/etc/quiltrc push' exited with 0  Output:
> > Applying patch 0009-Change-order-of-CFLAGS.patch
> > patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required
> > by patch)
> > Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce
> > with -f)
> > 
> > which is a symptom of this issue (libattr-native is a dependency of
> > patch-native).
> > 
> 
> ah so this was the problem I wondered

Its one problem we're seeing, I'm still not sure those two other
patches of yours apply cleanly though... :/

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] staging: Handle races between binaries and their libs
  2020-02-14 14:50   ` Richard Purdie
@ 2020-02-14 15:44     ` Khem Raj
  0 siblings, 0 replies; 6+ messages in thread
From: Khem Raj @ 2020-02-14 15:44 UTC (permalink / raw)
  To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer

On Fri, Feb 14, 2020 at 6:51 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2020-02-14 at 06:46 -0800, Khem Raj wrote:
> > On Fri, Feb 14, 2020 at 5:03 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > There is a long standing issue where a binary could be installed
> > > into the
> > > sysroot before its library dependencies. We've always argued
> > > nothing should
> > > use the binary until its been installed by a dependency but there
> > > are issues
> > > around binaries which conflict with the host system, for example
> > > patch,
> > > python3, gzip and more.
> > >
> > > With the recent patch changes we've see issues like:
> > > ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error:
> > > 'quilt --quiltrc /home/pokybuild/yocto-
> > > worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-
> > > linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-
> > > native/etc/quiltrc push' exited with 0  Output:
> > > Applying patch 0009-Change-order-of-CFLAGS.patch
> > > patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required
> > > by patch)
> > > Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce
> > > with -f)
> > >
> > > which is a symptom of this issue (libattr-native is a dependency of
> > > patch-native).
> > >
> >
> > ah so this was the problem I wondered
>
> Its one problem we're seeing, I'm still not sure those two other
> patches of yours apply cleanly though... :/
>

I have resent the patches for musl upgrade using git send-mail instead
of ./scripts/send-pull-request, can you try these from mailing list ?

> Cheers,
>
> Richard
>


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] staging: Handle races between binaries and their libs
  2020-02-14 13:02 [PATCH] staging: Handle races between binaries and their libs Richard Purdie
  2020-02-14 14:46 ` Khem Raj
@ 2020-02-14 17:01 ` Peter Kjellerstedt
  2020-02-14 17:33   ` Richard Purdie
  1 sibling, 1 reply; 6+ messages in thread
From: Peter Kjellerstedt @ 2020-02-14 17:01 UTC (permalink / raw)
  To: Richard Purdie; +Cc: openembedded-core

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org <openembedded-core-
> bounces@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 14 februari 2020 14:03
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] staging: Handle races between binaries and
> their libs
> 
> There is a long standing issue where a binary could be installed into the
> sysroot before its library dependencies. We've always argued nothing should
> use the binary until its been installed by a dependency but there are issues

Change "its" to "it has".

> around binaries which conflict with the host system, for example patch,
> python3, gzip and more.
> 
> With the recent patch changes we've see issues like:

Change "see" to "seen".

> ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0  Output:
> Applying patch 0009-Change-order-of-CFLAGS.patch
> patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
> Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)
> 
> which is a symptom of this issue (libattr-native is a dependency of patch-native).
> 
> There are other ways to fix this such as disabling libattr in patch, installing
> patch to a subdirectory and requiring PATH manipulation and so on.
> 
> We can simply fix the staging code to handle /bin/ after everything else so lets

Change "lets" to "let's".

> do that and avoid all these other complications.

Shouldn't /sbin/ be treated the same way as /bin/?

//Peter

> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/staging.bbclass | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
> index 4dd2ed01010..530e23b1853 100644
> --- a/meta/classes/staging.bbclass
> +++ b/meta/classes/staging.bbclass
> @@ -470,6 +470,7 @@ python extend_recipe_sysroot() {
>          elif os.path.lexists(depdir + "/" + c):
>              os.unlink(depdir + "/" + c)
> 
> +    binfiles = {}
>      # Now handle installs
>      for dep in configuredeps:
>          c = setscenedeps[dep][0]
> @@ -562,7 +563,16 @@ python extend_recipe_sysroot() {
>                      if l.endswith("/"):
>                          staging_copydir(l, targetdir, dest, seendirs)
>                          continue
> -                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
> +                    if "/bin/" in l:
> +                        # defer /bin/* files until last in case they need libs
> +                        binfiles[l] = (targetdir, dest)
> +                    else:
> +                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
> +
> +    # Handle deferred binfiles
> +    for l in binfiles:
> +        (targetdir, dest) = binfiles[l]
> +        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
> 
>      bb.note("Installed into sysroot: %s" % str(msg_adding))
>      bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
> --
> 2.20.1

//Peter



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] staging: Handle races between binaries and their libs
  2020-02-14 17:01 ` Peter Kjellerstedt
@ 2020-02-14 17:33   ` Richard Purdie
  0 siblings, 0 replies; 6+ messages in thread
From: Richard Purdie @ 2020-02-14 17:33 UTC (permalink / raw)
  To: Peter Kjellerstedt; +Cc: openembedded-core

On Fri, 2020-02-14 at 17:01 +0000, Peter Kjellerstedt wrote:
> Shouldn't /sbin/ be treated the same way as /bin/?

I guess so, yes. I've done that.

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-02-14 17:33 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-14 13:02 [PATCH] staging: Handle races between binaries and their libs Richard Purdie
2020-02-14 14:46 ` Khem Raj
2020-02-14 14:50   ` Richard Purdie
2020-02-14 15:44     ` Khem Raj
2020-02-14 17:01 ` Peter Kjellerstedt
2020-02-14 17:33   ` Richard Purdie

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.