All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
@ 2018-11-02 14:39 matthias.schiffer
  2018-11-02 15:19 ` Peter Kjellerstedt
  0 siblings, 1 reply; 8+ messages in thread
From: matthias.schiffer @ 2018-11-02 14:39 UTC (permalink / raw)
  To: openembedded-core

From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>

Until recent versions of findutils, -ignore_readdir_race did not work
correctly with -delete. This led to spurious build failures when files
disappeared; specifically this was seen due to do_configure and
do_populate_lic running concurrently for packages with ${B} == ${WORKDIR}:

   find: '.../sstate-build-populate_lic': No such file or directory

As the find command of the host system is used here, we can't ensure that
the used version contains the fix. Work around the issue by passing the
output of find to 'rm -f' instead of using -delete.

Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
---

It would be great if this patch could be backported to all versions back to
Rocko, as our current setup is still based on Rocko.

 meta/classes/base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index df11c8b270..5ffd8396e3 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -303,7 +303,7 @@ base_do_configure() {
 			if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile -o -e makefile -o -e GNUmakefile \) ]; then
 				oe_runmake clean
 			fi
-			find ${B} -ignore_readdir_race -name \*.la -delete
+			find ${B} -ignore_readdir_race -name \*.la -type f -print0 | xargs -0 rm -f
 		fi
 	fi
 	if [ -n "${CONFIGURESTAMPFILE}" ]; then
-- 
2.17.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-02 14:39 [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete' matthias.schiffer
@ 2018-11-02 15:19 ` Peter Kjellerstedt
  2018-11-02 15:43   ` Richard Purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Peter Kjellerstedt @ 2018-11-02 15:19 UTC (permalink / raw)
  To: matthias.schiffer; +Cc: openembedded-core

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org <openembedded-
> core-bounces@lists.openembedded.org> On Behalf Of
> matthias.schiffer@ew.tq-group.com
> Sent: den 2 november 2018 15:39
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> ignore_readdir_race -delete'
> 
> From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> 
> Until recent versions of findutils, -ignore_readdir_race did not work
> correctly with -delete. This led to spurious build failures when files
> disappeared; specifically this was seen due to do_configure and
> do_populate_lic running concurrently for packages with ${B} == ${WORKDIR}:
> 
>    find: '.../sstate-build-populate_lic': No such file or directory
> 
> As the find command of the host system is used here, we can't ensure that
> the used version contains the fix. Work around the issue by passing the
> output of find to 'rm -f' instead of using -delete.
> 
> Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> ---
> 
> It would be great if this patch could be backported to all versions
> back to Rocko, as our current setup is still based on Rocko.
> 
>  meta/classes/base.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index df11c8b270..5ffd8396e3 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -303,7 +303,7 @@ base_do_configure() {
>  			if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile -o -e makefile -o -e GNUmakefile \) ]; then
>  				oe_runmake clean
>  			fi

You might want to include a comment here explaining why rm -f is used 
instead of -delete, or someone might come along and optimize the code...

> -			find ${B} -ignore_readdir_race -name \*.la -delete
> +			find ${B} -ignore_readdir_race -name \*.la -type f -print0 | xargs -0 rm -f
>  		fi
>  	fi
>  	if [ -n "${CONFIGURESTAMPFILE}" ]; then
> --
> 2.17.1

//Peter



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-02 15:19 ` Peter Kjellerstedt
@ 2018-11-02 15:43   ` Richard Purdie
  2018-11-03  3:28     ` Andre McCurdy
  0 siblings, 1 reply; 8+ messages in thread
From: Richard Purdie @ 2018-11-02 15:43 UTC (permalink / raw)
  To: Peter Kjellerstedt, matthias.schiffer; +Cc: openembedded-core

On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: openembedded-core-bounces@lists.openembedded.org
> > <openembedded-
> > core-bounces@lists.openembedded.org> On Behalf Of
> > matthias.schiffer@ew.tq-group.com
> > Sent: den 2 november 2018 15:39
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > ignore_readdir_race -delete'
> > 
> > From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> > 
> > Until recent versions of findutils, -ignore_readdir_race did not
> > work
> > correctly with -delete. This led to spurious build failures when
> > files
> > disappeared; specifically this was seen due to do_configure and
> > do_populate_lic running concurrently for packages with ${B} ==
> > ${WORKDIR}:
> > 
> >    find: '.../sstate-build-populate_lic': No such file or directory
> > 
> > As the find command of the host system is used here, we can't
> > ensure that
> > the used version contains the fix. Work around the issue by passing
> > the
> > output of find to 'rm -f' instead of using -delete.
> > 
> > Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com
> > >
> > ---
> > 
> > It would be great if this patch could be backported to all versions
> > back to Rocko, as our current setup is still based on Rocko.
> > 
> >  meta/classes/base.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> > index df11c8b270..5ffd8396e3 100644
> > --- a/meta/classes/base.bbclass
> > +++ b/meta/classes/base.bbclass
> > @@ -303,7 +303,7 @@ base_do_configure() {
> >  			if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile
> > -o -e makefile -o -e GNUmakefile \) ]; then
> >  				oe_runmake clean
> >  			fi
> 
> You might want to include a comment here explaining why rm -f is
> used instead of -delete, or someone might come along and optimize the
> code...

We should detail which versions of findutils have the issue as
ultimately it would be nice to use the simpler command. I'm curious how
many of the distros we support have the problem.

Cheers,

Richard



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-02 15:43   ` Richard Purdie
@ 2018-11-03  3:28     ` Andre McCurdy
  2018-11-05  8:36       ` Matthias Schiffer
  0 siblings, 1 reply; 8+ messages in thread
From: Andre McCurdy @ 2018-11-03  3:28 UTC (permalink / raw)
  To: Richard Purdie; +Cc: Peter Kjellerstedt, OE Core mailing list

On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > -----Original Message-----
> > > From: openembedded-core-bounces@lists.openembedded.org
> > > <openembedded-
> > > core-bounces@lists.openembedded.org> On Behalf Of
> > > matthias.schiffer@ew.tq-group.com
> > > Sent: den 2 november 2018 15:39
> > > To: openembedded-core@lists.openembedded.org
> > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > ignore_readdir_race -delete'
> > >
> > > From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> > >
> > > Until recent versions of findutils, -ignore_readdir_race did not
> > > work
> > > correctly with -delete. This led to spurious build failures when
> > > files
> > > disappeared; specifically this was seen due to do_configure and
> > > do_populate_lic running concurrently for packages with ${B} ==
> > > ${WORKDIR}:
> > >
> > >    find: '.../sstate-build-populate_lic': No such file or directory
> > >
> > > As the find command of the host system is used here, we can't
> > > ensure that
> > > the used version contains the fix. Work around the issue by passing
> > > the
> > > output of find to 'rm -f' instead of using -delete.
> > >
> > > Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com
> > > >
> > > ---
> > >
> > > It would be great if this patch could be backported to all versions
> > > back to Rocko, as our current setup is still based on Rocko.
> > >
> > >  meta/classes/base.bbclass | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> > > index df11c8b270..5ffd8396e3 100644
> > > --- a/meta/classes/base.bbclass
> > > +++ b/meta/classes/base.bbclass
> > > @@ -303,7 +303,7 @@ base_do_configure() {
> > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile
> > > -o -e makefile -o -e GNUmakefile \) ]; then
> > >                             oe_runmake clean
> > >                     fi
> >
> > You might want to include a comment here explaining why rm -f is
> > used instead of -delete, or someone might come along and optimize the
> > code...
>
> We should detail which versions of findutils have the issue as
> ultimately it would be nice to use the simpler command. I'm curious how
> many of the distros we support have the problem.

There's a test case in the patch proposed to fix the issue:

  https://savannah.gnu.org/bugs/?52981
  https://savannah.gnu.org/bugs/download.php?file_id=43119

Regarding the proposed patch for base.bbclass, xargs should generally
always be called with --no-run-if-empty, unless there's a clear reason
not to.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-03  3:28     ` Andre McCurdy
@ 2018-11-05  8:36       ` Matthias Schiffer
  2018-11-06  8:23         ` Matthias Schiffer
  0 siblings, 1 reply; 8+ messages in thread
From: Matthias Schiffer @ 2018-11-05  8:36 UTC (permalink / raw)
  To: Andre McCurdy, Richard Purdie; +Cc: Peter Kjellerstedt, OE Core mailing list

On Fri, 2018-11-02 at 20:28 -0700, Andre McCurdy wrote:
> On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > 
> > On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > > -----Original Message-----
> > > > From: openembedded-core-bounces@lists.openembedded.org
> > > > <openembedded-
> > > > core-bounces@lists.openembedded.org> On Behalf Of
> > > > matthias.schiffer@ew.tq-group.com
> > > > Sent: den 2 november 2018 15:39
> > > > To: openembedded-core@lists.openembedded.org
> > > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > > ignore_readdir_race -delete'
> > > > 
> > > > From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> > > > 
> > > > Until recent versions of findutils, -ignore_readdir_race did
> > > > not
> > > > work
> > > > correctly with -delete. This led to spurious build failures
> > > > when
> > > > files
> > > > disappeared; specifically this was seen due to do_configure and
> > > > do_populate_lic running concurrently for packages with ${B} ==
> > > > ${WORKDIR}:
> > > > 
> > > >    find: '.../sstate-build-populate_lic': No such file or
> > > > directory
> > > > 
> > > > As the find command of the host system is used here, we can't
> > > > ensure that
> > > > the used version contains the fix. Work around the issue by
> > > > passing
> > > > the
> > > > output of find to 'rm -f' instead of using -delete.
> > > > 
> > > > Signed-off-by: Matthias Schiffer <
> > > > matthias.schiffer@ew.tq-group.com
> > > > > 
> > > > 
> > > > ---
> > > > 
> > > > It would be great if this patch could be backported to all
> > > > versions
> > > > back to Rocko, as our current setup is still based on Rocko.
> > > > 
> > > >  meta/classes/base.bbclass | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/meta/classes/base.bbclass
> > > > b/meta/classes/base.bbclass
> > > > index df11c8b270..5ffd8396e3 100644
> > > > --- a/meta/classes/base.bbclass
> > > > +++ b/meta/classes/base.bbclass
> > > > @@ -303,7 +303,7 @@ base_do_configure() {
> > > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e
> > > > Makefile
> > > > -o -e makefile -o -e GNUmakefile \) ]; then
> > > >                             oe_runmake clean
> > > >                     fi
> > > 
> > > You might want to include a comment here explaining why rm -f is
> > > used instead of -delete, or someone might come along and optimize
> > > the
> > > code...
> > 
> > We should detail which versions of findutils have the issue as
> > ultimately it would be nice to use the simpler command. I'm curious
> > how
> > many of the distros we support have the problem.
> 
> There's a test case in the patch proposed to fix the issue:
> 
>   https://savannah.gnu.org/bugs/?52981
>   https://savannah.gnu.org/bugs/download.php?file_id=43119
> 
> Regarding the proposed patch for base.bbclass, xargs should generally
> always be called with --no-run-if-empty, unless there's a clear
> reason
> not to.

Do we generally assume that we have a GNU xargs with --no-run-if-empty
support? Calling rm -f without further arguments at least doesn't cause
an error, so --no-run-if-empty is only a minor optimization in this
case.

I'll send an updated patch with the proposed changes later today.

Kinds regards,
Matthias




^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-05  8:36       ` Matthias Schiffer
@ 2018-11-06  8:23         ` Matthias Schiffer
  2018-11-06  8:25           ` Andre McCurdy
  0 siblings, 1 reply; 8+ messages in thread
From: Matthias Schiffer @ 2018-11-06  8:23 UTC (permalink / raw)
  To: Andre McCurdy, Richard Purdie; +Cc: Peter Kjellerstedt, OE Core mailing list

On Mon, 2018-11-05 at 09:36 +0100, Matthias Schiffer wrote:
> On Fri, 2018-11-02 at 20:28 -0700, Andre McCurdy wrote:
> > On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > 
> > > On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > > > -----Original Message-----
> > > > > From: openembedded-core-bounces@lists.openembedded.org
> > > > > <openembedded-
> > > > > core-bounces@lists.openembedded.org> On Behalf Of
> > > > > matthias.schiffer@ew.tq-group.com
> > > > > Sent: den 2 november 2018 15:39
> > > > > To: openembedded-core@lists.openembedded.org
> > > > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > > > ignore_readdir_race -delete'
> > > > > 
> > > > > From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> > > > > 
> > > > > Until recent versions of findutils, -ignore_readdir_race did
> > > > > not
> > > > > work
> > > > > correctly with -delete. This led to spurious build failures
> > > > > when
> > > > > files
> > > > > disappeared; specifically this was seen due to do_configure
> > > > > and
> > > > > do_populate_lic running concurrently for packages with ${B}
> > > > > ==
> > > > > ${WORKDIR}:
> > > > > 
> > > > >    find: '.../sstate-build-populate_lic': No such file or
> > > > > directory
> > > > > 
> > > > > As the find command of the host system is used here, we can't
> > > > > ensure that
> > > > > the used version contains the fix. Work around the issue by
> > > > > passing
> > > > > the
> > > > > output of find to 'rm -f' instead of using -delete.
> > > > > 
> > > > > Signed-off-by: Matthias Schiffer <
> > > > > matthias.schiffer@ew.tq-group.com
> > > > > > 
> > > > > 
> > > > > ---
> > > > > 
> > > > > It would be great if this patch could be backported to all
> > > > > versions
> > > > > back to Rocko, as our current setup is still based on Rocko.
> > > > > 
> > > > >  meta/classes/base.bbclass | 2 +-
> > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/meta/classes/base.bbclass
> > > > > b/meta/classes/base.bbclass
> > > > > index df11c8b270..5ffd8396e3 100644
> > > > > --- a/meta/classes/base.bbclass
> > > > > +++ b/meta/classes/base.bbclass
> > > > > @@ -303,7 +303,7 @@ base_do_configure() {
> > > > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e
> > > > > Makefile
> > > > > -o -e makefile -o -e GNUmakefile \) ]; then
> > > > >                             oe_runmake clean
> > > > >                     fi
> > > > 
> > > > You might want to include a comment here explaining why rm -f
> > > > is
> > > > used instead of -delete, or someone might come along and
> > > > optimize
> > > > the
> > > > code...
> > > 
> > > We should detail which versions of findutils have the issue as
> > > ultimately it would be nice to use the simpler command. I'm
> > > curious
> > > how
> > > many of the distros we support have the problem.
> > 
> > There's a test case in the patch proposed to fix the issue:
> > 
> >   https://savannah.gnu.org/bugs/?52981
> >   https://savannah.gnu.org/bugs/download.php?file_id=43119
> > 
> > Regarding the proposed patch for base.bbclass, xargs should
> > generally
> > always be called with --no-run-if-empty, unless there's a clear
> > reason
> > not to.
> 
> Do we generally assume that we have a GNU xargs with --no-run-if-
> empty
> support? Calling rm -f without further arguments at least doesn't
> cause
> an error, so --no-run-if-empty is only a minor optimization in this
> case.


I've grepped bitbake, openembedded-core and meta-openembedded for --no-
run-if-empty, and there is only a single instance in the whole code
base - which is conditionalized so that the option is not passed on
Darwin. I take from this that --no-run-if-empty is not acceptable in
OpenEmbedded to ensure portability.

> 
> I'll send an updated patch with the proposed changes later today.
> 
> Kinds regards,
> Matthias
> 



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-06  8:23         ` Matthias Schiffer
@ 2018-11-06  8:25           ` Andre McCurdy
  2018-11-06  8:38             ` Matthias Schiffer
  0 siblings, 1 reply; 8+ messages in thread
From: Andre McCurdy @ 2018-11-06  8:25 UTC (permalink / raw)
  To: matthias.schiffer; +Cc: Peter Kjellerstedt, OE Core mailing list

On Tue, Nov 6, 2018 at 12:23 AM Matthias Schiffer
<matthias.schiffer@ew.tq-group.com> wrote:
>
> On Mon, 2018-11-05 at 09:36 +0100, Matthias Schiffer wrote:
> > On Fri, 2018-11-02 at 20:28 -0700, Andre McCurdy wrote:
> > > On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > >
> > > > On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > > > > -----Original Message-----
> > > > > > From: openembedded-core-bounces@lists.openembedded.org
> > > > > > <openembedded-
> > > > > > core-bounces@lists.openembedded.org> On Behalf Of
> > > > > > matthias.schiffer@ew.tq-group.com
> > > > > > Sent: den 2 november 2018 15:39
> > > > > > To: openembedded-core@lists.openembedded.org
> > > > > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > > > > ignore_readdir_race -delete'
> > > > > >
> > > > > > From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> > > > > >
> > > > > > Until recent versions of findutils, -ignore_readdir_race did
> > > > > > not
> > > > > > work
> > > > > > correctly with -delete. This led to spurious build failures
> > > > > > when
> > > > > > files
> > > > > > disappeared; specifically this was seen due to do_configure
> > > > > > and
> > > > > > do_populate_lic running concurrently for packages with ${B}
> > > > > > ==
> > > > > > ${WORKDIR}:
> > > > > >
> > > > > >    find: '.../sstate-build-populate_lic': No such file or
> > > > > > directory
> > > > > >
> > > > > > As the find command of the host system is used here, we can't
> > > > > > ensure that
> > > > > > the used version contains the fix. Work around the issue by
> > > > > > passing
> > > > > > the
> > > > > > output of find to 'rm -f' instead of using -delete.
> > > > > >
> > > > > > Signed-off-by: Matthias Schiffer <
> > > > > > matthias.schiffer@ew.tq-group.com
> > > > > > >
> > > > > >
> > > > > > ---
> > > > > >
> > > > > > It would be great if this patch could be backported to all
> > > > > > versions
> > > > > > back to Rocko, as our current setup is still based on Rocko.
> > > > > >
> > > > > >  meta/classes/base.bbclass | 2 +-
> > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/meta/classes/base.bbclass
> > > > > > b/meta/classes/base.bbclass
> > > > > > index df11c8b270..5ffd8396e3 100644
> > > > > > --- a/meta/classes/base.bbclass
> > > > > > +++ b/meta/classes/base.bbclass
> > > > > > @@ -303,7 +303,7 @@ base_do_configure() {
> > > > > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e
> > > > > > Makefile
> > > > > > -o -e makefile -o -e GNUmakefile \) ]; then
> > > > > >                             oe_runmake clean
> > > > > >                     fi
> > > > >
> > > > > You might want to include a comment here explaining why rm -f
> > > > > is
> > > > > used instead of -delete, or someone might come along and
> > > > > optimize
> > > > > the
> > > > > code...
> > > >
> > > > We should detail which versions of findutils have the issue as
> > > > ultimately it would be nice to use the simpler command. I'm
> > > > curious
> > > > how
> > > > many of the distros we support have the problem.
> > >
> > > There's a test case in the patch proposed to fix the issue:
> > >
> > >   https://savannah.gnu.org/bugs/?52981
> > >   https://savannah.gnu.org/bugs/download.php?file_id=43119
> > >
> > > Regarding the proposed patch for base.bbclass, xargs should
> > > generally
> > > always be called with --no-run-if-empty, unless there's a clear
> > > reason
> > > not to.
> >
> > Do we generally assume that we have a GNU xargs with --no-run-if-
> > empty
> > support? Calling rm -f without further arguments at least doesn't
> > cause
> > an error, so --no-run-if-empty is only a minor optimization in this
> > case.
>
>
> I've grepped bitbake, openembedded-core and meta-openembedded for --no-
> run-if-empty, and there is only a single instance in the whole code
> base - which is conditionalized so that the option is not passed on
> Darwin. I take from this that --no-run-if-empty is not acceptable in
> OpenEmbedded to ensure portability.

Did you grep for the short form too, ie -r ?

> >
> > I'll send an updated patch with the proposed changes later today.
> >
> > Kinds regards,
> > Matthias
> >
>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete'
  2018-11-06  8:25           ` Andre McCurdy
@ 2018-11-06  8:38             ` Matthias Schiffer
  0 siblings, 0 replies; 8+ messages in thread
From: Matthias Schiffer @ 2018-11-06  8:38 UTC (permalink / raw)
  To: Andre McCurdy; +Cc: Peter Kjellerstedt, OE Core mailing list

On Tue, 2018-11-06 at 00:25 -0800, Andre McCurdy wrote:
> On Tue, Nov 6, 2018 at 12:23 AM Matthias Schiffer
> <matthias.schiffer@ew.tq-group.com> wrote:
> > 
> > On Mon, 2018-11-05 at 09:36 +0100, Matthias Schiffer wrote:
> > > On Fri, 2018-11-02 at 20:28 -0700, Andre McCurdy wrote:
> > > > On Fri, Nov 2, 2018 at 8:43 AM Richard Purdie
> > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > 
> > > > > On Fri, 2018-11-02 at 15:19 +0000, Peter Kjellerstedt wrote:
> > > > > > > -----Original Message-----
> > > > > > > From: openembedded-core-bounces@lists.openembedded.org
> > > > > > > <openembedded-
> > > > > > > core-bounces@lists.openembedded.org> On Behalf Of
> > > > > > > matthias.schiffer@ew.tq-group.com
> > > > > > > Sent: den 2 november 2018 15:39
> > > > > > > To: openembedded-core@lists.openembedded.org
> > > > > > > Subject: [OE-core] [PATCH] base.bbclass: avoid 'find -
> > > > > > > ignore_readdir_race -delete'
> > > > > > > 
> > > > > > > From: Matthias Schiffer <
> > > > > > > matthias.schiffer@ew.tq-group.com>
> > > > > > > 
> > > > > > > Until recent versions of findutils, -ignore_readdir_race
> > > > > > > did
> > > > > > > not
> > > > > > > work
> > > > > > > correctly with -delete. This led to spurious build
> > > > > > > failures
> > > > > > > when
> > > > > > > files
> > > > > > > disappeared; specifically this was seen due to
> > > > > > > do_configure
> > > > > > > and
> > > > > > > do_populate_lic running concurrently for packages with
> > > > > > > ${B}
> > > > > > > ==
> > > > > > > ${WORKDIR}:
> > > > > > > 
> > > > > > >    find: '.../sstate-build-populate_lic': No such file or
> > > > > > > directory
> > > > > > > 
> > > > > > > As the find command of the host system is used here, we
> > > > > > > can't
> > > > > > > ensure that
> > > > > > > the used version contains the fix. Work around the issue
> > > > > > > by
> > > > > > > passing
> > > > > > > the
> > > > > > > output of find to 'rm -f' instead of using -delete.
> > > > > > > 
> > > > > > > Signed-off-by: Matthias Schiffer <
> > > > > > > matthias.schiffer@ew.tq-group.com
> > > > > > > > 
> > > > > > > 
> > > > > > > ---
> > > > > > > 
> > > > > > > It would be great if this patch could be backported to
> > > > > > > all
> > > > > > > versions
> > > > > > > back to Rocko, as our current setup is still based on
> > > > > > > Rocko.
> > > > > > > 
> > > > > > >  meta/classes/base.bbclass | 2 +-
> > > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/meta/classes/base.bbclass
> > > > > > > b/meta/classes/base.bbclass
> > > > > > > index df11c8b270..5ffd8396e3 100644
> > > > > > > --- a/meta/classes/base.bbclass
> > > > > > > +++ b/meta/classes/base.bbclass
> > > > > > > @@ -303,7 +303,7 @@ base_do_configure() {
> > > > > > >                     if [ "${CLEANBROKEN}" != "1" -a \( -e
> > > > > > > Makefile
> > > > > > > -o -e makefile -o -e GNUmakefile \) ]; then
> > > > > > >                             oe_runmake clean
> > > > > > >                     fi
> > > > > > 
> > > > > > You might want to include a comment here explaining why rm
> > > > > > -f
> > > > > > is
> > > > > > used instead of -delete, or someone might come along and
> > > > > > optimize
> > > > > > the
> > > > > > code...
> > > > > 
> > > > > We should detail which versions of findutils have the issue
> > > > > as
> > > > > ultimately it would be nice to use the simpler command. I'm
> > > > > curious
> > > > > how
> > > > > many of the distros we support have the problem.
> > > > 
> > > > There's a test case in the patch proposed to fix the issue:
> > > > 
> > > >   https://savannah.gnu.org/bugs/?52981
> > > >   https://savannah.gnu.org/bugs/download.php?file_id=43119
> > > > 
> > > > Regarding the proposed patch for base.bbclass, xargs should
> > > > generally
> > > > always be called with --no-run-if-empty, unless there's a clear
> > > > reason
> > > > not to.
> > > 
> > > Do we generally assume that we have a GNU xargs with --no-run-if-
> > > empty
> > > support? Calling rm -f without further arguments at least doesn't
> > > cause
> > > an error, so --no-run-if-empty is only a minor optimization in
> > > this
> > > case.
> > 
> > 
> > I've grepped bitbake, openembedded-core and meta-openembedded for
> > --no-
> > run-if-empty, and there is only a single instance in the whole code
> > base - which is conditionalized so that the option is not passed on
> > Darwin. I take from this that --no-run-if-empty is not acceptable
> > in
> > OpenEmbedded to ensure portability.
> 
> Did you grep for the short form too, ie -r ?

Thanks for the pointer; including the short form, I've found a second
instance. Still, in the vast majority of xargs commands it is avoided,
so I guess I should the same to stay consistent with the existing
scripts.

> 
> > > 
> > > I'll send an updated patch with the proposed changes later today.
> > > 
> > > Kinds regards,
> > > Matthias
> > > 



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2018-11-06  8:38 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-02 14:39 [PATCH] base.bbclass: avoid 'find -ignore_readdir_race -delete' matthias.schiffer
2018-11-02 15:19 ` Peter Kjellerstedt
2018-11-02 15:43   ` Richard Purdie
2018-11-03  3:28     ` Andre McCurdy
2018-11-05  8:36       ` Matthias Schiffer
2018-11-06  8:23         ` Matthias Schiffer
2018-11-06  8:25           ` Andre McCurdy
2018-11-06  8:38             ` Matthias Schiffer

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.