[PATCH v2] base.bbclass: avoid 'find -ignore_readdir_race -delete'

[PATCH v2] base.bbclass: avoid 'find -ignore_readdir_race -delete'

[matthias.schiffer]

From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>

Due to a bug in find [1], -ignore_readdir_race does not work correctly with
-delete. This can lead to spurious build failures when files disappear
while such a command is running; specifically this was seen in the case of
do_configure and do_populate_lic running concurrently for packages
with ${B} == ${WORKDIR}:

   find: '.../sstate-build-populate_lic': No such file or directory

While the issue is fixed in the findutils git master, the find command of
the host system is called here, so we can't ensure that the used version
contains the fix. Many common distros have not updated to a recent enough
findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is still
affected).

Work around the issue by passing the output of find to 'rm -f' instead of
using -delete.

[1] https://savannah.gnu.org/bugs/?52981

Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
---

It would be great if this patch could be backported to all versions back to
Rocko, as our current setup is still based on Rocko.

v2: add comment, extend commit message


 meta/classes/base.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index df11c8b270..9efb06de5f 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -303,7 +303,9 @@ base_do_configure() {
 			if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile -o -e makefile -o -e GNUmakefile \) ]; then
 				oe_runmake clean
 			fi
-			find ${B} -ignore_readdir_race -name \*.la -delete
+			# -ignore_readdir_race does not work correctly with -delete;
+			# use xargs to avoid spurious build failures
+			find ${B} -ignore_readdir_race -name \*.la -type f -print0 | xargs -0 rm -f
 		fi
 	fi
 	if [ -n "${CONFIGURESTAMPFILE}" ]; then
-- 
2.17.1

Re: [PATCH v2] base.bbclass: avoid 'find -ignore_readdir_race -delete'

[Matthias Schiffer]

On Tue, 2018-11-06 at 09:56 +0100, matthias.schiffer@ew.tq-group.com
wrote:
> From: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> 
> Due to a bug in find [1], -ignore_readdir_race does not work
> correctly with
> -delete. This can lead to spurious build failures when files
> disappear
> while such a command is running; specifically this was seen in the
> case of
> do_configure and do_populate_lic running concurrently for packages
> with ${B} == ${WORKDIR}:
> 
>    find: '.../sstate-build-populate_lic': No such file or directory
> 
> While the issue is fixed in the findutils git master, the find
> command of
> the host system is called here, so we can't ensure that the used
> version
> contains the fix. Many common distros have not updated to a recent
> enough
> findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is
> still
> affected).
> 
> Work around the issue by passing the output of find to 'rm -f'
> instead of
> using -delete.
> 
> [1] https://savannah.gnu.org/bugs/?52981
> 
> Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
> ---
> 
> It would be great if this patch could be backported to all versions
> back to
> Rocko, as our current setup is still based on Rocko.

Is there any chance we can get this backported?

Thanks,
Matthias


> 
> v2: add comment, extend commit message
> 
> 
>  meta/classes/base.bbclass | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index df11c8b270..9efb06de5f 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -303,7 +303,9 @@ base_do_configure() {
>  			if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile
> -o -e makefile -o -e GNUmakefile \) ]; then
>  				oe_runmake clean
>  			fi
> -			find ${B} -ignore_readdir_race -name \*.la
> -delete
> +			# -ignore_readdir_race does not work correctly
> with -delete;
> +			# use xargs to avoid spurious build failures
> +			find ${B} -ignore_readdir_race -name \*.la
> -type f -print0 | xargs -0 rm -f
>  		fi
>  	fi
>  	if [ -n "${CONFIGURESTAMPFILE}" ]; then

Re: [PATCH v2] base.bbclass: avoid 'find -ignore_readdir_race -delete'

[Matthias Schiffer]

On Fri, 2018-11-23 at 11:35 +0100, Matthias Schiffer wrote:
> On Tue, 2018-11-06 at 09:56 +0100, matthias.schiffer@ew.tq-group.com
> wrote:
[...]
> > ---
> > 
> > It would be great if this patch could be backported to all versions
> > back to
> > Rocko, as our current setup is still based on Rocko.
> 
> Is there any chance we can get this backported?
> 
> Thanks,
> Matthias

Thanks, I saw this is in Sumo now. Can we get it into Rocko as well?


> 
> 
> > 
> > v2: add comment, extend commit message
> > 
> > 
> >  meta/classes/base.bbclass | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> > index df11c8b270..9efb06de5f 100644
> > --- a/meta/classes/base.bbclass
> > +++ b/meta/classes/base.bbclass
> > @@ -303,7 +303,9 @@ base_do_configure() {
> >  			if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile
> > -o -e makefile -o -e GNUmakefile \) ]; then
> >  				oe_runmake clean
> >  			fi
> > -			find ${B} -ignore_readdir_race -name \*.la
> > -delete
> > +			# -ignore_readdir_race does not work correctly
> > with -delete;
> > +			# use xargs to avoid spurious build failures
> > +			find ${B} -ignore_readdir_race -name \*.la
> > -type f -print0 | xargs -0 rm -f
> >  		fi
> >  	fi
> >  	if [ -n "${CONFIGURESTAMPFILE}" ]; then