[PATCH] rm_work.bbclass: use HOSTTOOLS 'rm' binary exclusively

[PATCH] rm_work.bbclass: use HOSTTOOLS 'rm' binary exclusively

[Luis Martins]

The do_rm_work() task is using the first available 'rm' binary
available in PATH to remove files and folders.
However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE
contents, the function can be using the 'rm' binary available
in RECIPE_SYSROOT_NATIVE, a folder that will get removed.
This causes a sporadic race-condition when trying to access the
'rm' binary of a folder already deleted.
Solve this by exclusively using the HOSTTOOLS 'rm' binary, as
this folder will not get removed.

Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com>
---
 meta/classes/rm_work.bbclass | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass
index 1f28bc7187..8b5fe1b808 100644
--- a/meta/classes/rm_work.bbclass
+++ b/meta/classes/rm_work.bbclass
@@ -33,6 +33,13 @@ BB_SCHEDULER ?= "completion"
 BB_TASK_IONICE_LEVEL:task-rm_work = "3.0"
 
 do_rm_work () {
+    # Force using the HOSTTOOLS 'rm' - otherwise the SYSROOT_NATIVE 'rm' can be selected depending on PATH
+    # Avoids race-condition accessing 'rm' when deleting WORKDIR folders at the end of this function
+    RM_BIN="$(PATH=${HOSTTOOLS_DIR} command -v rm)"
+    if [ -z "${RM_BIN}" ]; then
+        bbfatal "Binary 'rm' not found in HOSTTOOLS_DIR, cannot remove WORKDIR data."
+    fi
+
     # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe.
     for p in ${RM_WORK_EXCLUDE}; do
         if [ "$p" = "${PN}" ]; then
@@ -79,7 +86,7 @@ do_rm_work () {
             # sstate version since otherwise we'd need to leave 'plaindirs' around
             # such as 'packages' and 'packages-split' and these can be large. No end
             # of chain tasks depend directly on do_package anymore.
-            rm -f -- $i;
+            "${RM_BIN}" -f -- $i;
             ;;
         *_setscene*)
             # Skip stamps which are already setscene versions
@@ -96,7 +103,7 @@ do_rm_work () {
                     ;;
                 esac
             done
-            rm -f -- $i
+            "${RM_BIN}" -f -- $i
         esac
     done
 
@@ -106,9 +113,9 @@ do_rm_work () {
         # Retain only logs and other files in temp, safely ignore
         # failures of removing pseudo folers on NFS2/3 server.
         if [ $dir = 'pseudo' ]; then
-            rm -rf -- $dir 2> /dev/null || true
+            "${RM_BIN}" -rf -- $dir 2> /dev/null || true
         elif ! echo "$excludes" | grep -q -w "$dir"; then
-            rm -rf -- $dir
+            "${RM_BIN}" -rf -- $dir
         fi
     done
 }
-- 
2.25.1

Re: [PATCH] rm_work.bbclass: use HOSTTOOLS 'rm' binary exclusively

[Luís Martins]

[-- Attachment #1: Type: text/plain, Size: 3008 bytes --]

Hi, I see this was accepted for master, is it okay to port this to dunfell
and kirkstone as well ?

Best regards,
Luís Martins


On Sat, 24 Dec 2022 at 01:49, Luis Martins <luis.pinto.martins@gmail.com>
wrote:

> The do_rm_work() task is using the first available 'rm' binary
> available in PATH to remove files and folders.
> However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE
> contents, the function can be using the 'rm' binary available
> in RECIPE_SYSROOT_NATIVE, a folder that will get removed.
> This causes a sporadic race-condition when trying to access the
> 'rm' binary of a folder already deleted.
> Solve this by exclusively using the HOSTTOOLS 'rm' binary, as
> this folder will not get removed.
>
> Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com>
> ---
>  meta/classes/rm_work.bbclass | 15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass
> index 1f28bc7187..8b5fe1b808 100644
> --- a/meta/classes/rm_work.bbclass
> +++ b/meta/classes/rm_work.bbclass
> @@ -33,6 +33,13 @@ BB_SCHEDULER ?= "completion"
>  BB_TASK_IONICE_LEVEL:task-rm_work = "3.0"
>
>  do_rm_work () {
> +    # Force using the HOSTTOOLS 'rm' - otherwise the SYSROOT_NATIVE 'rm'
> can be selected depending on PATH
> +    # Avoids race-condition accessing 'rm' when deleting WORKDIR folders
> at the end of this function
> +    RM_BIN="$(PATH=${HOSTTOOLS_DIR} command -v rm)"
> +    if [ -z "${RM_BIN}" ]; then
> +        bbfatal "Binary 'rm' not found in HOSTTOOLS_DIR, cannot remove
> WORKDIR data."
> +    fi
> +
>      # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe.
>      for p in ${RM_WORK_EXCLUDE}; do
>          if [ "$p" = "${PN}" ]; then
> @@ -79,7 +86,7 @@ do_rm_work () {
>              # sstate version since otherwise we'd need to leave
> 'plaindirs' around
>              # such as 'packages' and 'packages-split' and these can be
> large. No end
>              # of chain tasks depend directly on do_package anymore.
> -            rm -f -- $i;
> +            "${RM_BIN}" -f -- $i;
>              ;;
>          *_setscene*)
>              # Skip stamps which are already setscene versions
> @@ -96,7 +103,7 @@ do_rm_work () {
>                      ;;
>                  esac
>              done
> -            rm -f -- $i
> +            "${RM_BIN}" -f -- $i
>          esac
>      done
>
> @@ -106,9 +113,9 @@ do_rm_work () {
>          # Retain only logs and other files in temp, safely ignore
>          # failures of removing pseudo folers on NFS2/3 server.
>          if [ $dir = 'pseudo' ]; then
> -            rm -rf -- $dir 2> /dev/null || true
> +            "${RM_BIN}" -rf -- $dir 2> /dev/null || true
>          elif ! echo "$excludes" | grep -q -w "$dir"; then
> -            rm -rf -- $dir
> +            "${RM_BIN}" -rf -- $dir
>          fi
>      done
>  }
> --
> 2.25.1
>
>

[-- Attachment #2: Type: text/html, Size: 4169 bytes --]