bitbake-devel.lists.openembedded.org archive mirror
 help / color / mirror / Atom feed
From: Steve Sakoman <steve@sakoman.com>
To: bitbake-devel@lists.openembedded.org
Subject: [bitbake][dunfell][1.46][PATCH 1/7] runqueue: Fix multiconfig deferred task sstate validity caching issue
Date: Mon, 20 Feb 2023 12:25:17 -1000	[thread overview]
Message-ID: <f95ed69e6c91ca6aa18d6fa9a6ac6319035c4661.1676931822.git.steve@sakoman.com> (raw)
In-Reply-To: <cover.1676931822.git.steve@sakoman.com>

From: Richard Purdie <richard.purdie@linuxfoundation.org>

We were testing the validity of deferred tasks setscene status "up front" which
is very unlikely to succeed and leads to cache invalidation issues. With the
change to rebuild the deferred task list, this status becomes out of sync. The
result was tasks being executed when they should not have been leading to extra
work for the build unnecessarily.

Instead, don't process validity status for deferred tasks and assume their
data will become available. If it doesn't, this will now result in a build
error as the setscene task will fail and the main task will run instead.

In theory we could try and track the state changes in the deferred list and
re-test validity then but I'm not sure it is worth the effort when the other
code path and errors in setscene tasks will give a pretty good idea of what
is happening anyway.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit edcafac13b3b241b6687419e59018d21811507a1)
Signed-off-by: Fabio Berton <fabio.berton@criticaltechworks.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/runqueue.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 6cdc72a8..f82bc413 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -2084,8 +2084,6 @@ class RunQueueExecute:
                             logger.debug(1, "%s didn't become valid, skipping setscene" % nexttask)
                             self.sq_task_failoutright(nexttask)
                             return True
-                        else:
-                            self.sqdata.outrightfail.remove(nexttask)
                     if nexttask in self.sqdata.outrightfail:
                         logger.debug(2, 'No package found, so skipping setscene task %s', nexttask)
                         self.sq_task_failoutright(nexttask)
@@ -2843,6 +2841,8 @@ def update_scenequeue_data(tids, sqdata, rqdata, rq, cooker, stampcache, sqrq, s
             sqdata.stamppresent.remove(tid)
         if tid in sqdata.valid:
             sqdata.valid.remove(tid)
+        if tid in sqdata.outrightfail:
+            sqdata.outrightfail.remove(tid)
 
         (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
 
@@ -2887,10 +2887,10 @@ def update_scenequeue_data(tids, sqdata, rqdata, rq, cooker, stampcache, sqrq, s
             if tid in sqrq.scenequeue_covered:
                 continue
 
-            sqdata.outrightfail.add(tid)
-
             h = pending_hash_index(tid, rqdata)
             if h not in sqdata.hashes:
+                if tid in tids:
+                    sqdata.outrightfail.add(tid)
                 sqdata.hashes[h] = tid
             else:
                 sqrq.sq_deferred[tid] = sqdata.hashes[h]
-- 
2.34.1



  reply	other threads:[~2023-02-20 22:25 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-20 22:25 [bitbake][dunfell][1.46][PATCH 0/7] Patch review Steve Sakoman
2023-02-20 22:25 ` Steve Sakoman [this message]
2023-02-20 22:25 ` [bitbake][dunfell][1.46][PATCH 2/7] runqueue: Handle deferred task rehashing in multiconfig builds Steve Sakoman
2023-02-20 22:25 ` [bitbake][dunfell][1.46][PATCH 3/7] runqueue: Improve multiconfig deferred task issues Steve Sakoman
2023-02-20 22:25 ` [bitbake][dunfell][1.46][PATCH 4/7] runqueue: Avoid deadlock avoidance task graph corruption Steve Sakoman
2023-02-20 22:25 ` [bitbake][dunfell][1.46][PATCH 5/7] runqueue: Fix issues with multiconfig deferred task deadlock messages Steve Sakoman
2023-02-20 22:25 ` [bitbake][dunfell][1.46][PATCH 6/7] runqueue: Ensure deferred tasks are sorted by multiconfig Steve Sakoman
2023-02-20 22:25 ` [bitbake][dunfell][1.46][PATCH 7/7] cooker: Drop sre_constants usage Steve Sakoman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f95ed69e6c91ca6aa18d6fa9a6ac6319035c4661.1676931822.git.steve@sakoman.com \
    --to=steve@sakoman.com \
    --cc=bitbake-devel@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).