[PATCH 1/3] runqueue: Filter out multiconfig dependencies from BB_TASKDEPDATA

[PATCH 1/3] runqueue: Filter out multiconfig dependencies from BB_TASKDEPDATA

[Richard Purdie]

The consumers of BB_TASKDEPDATA in OE metadata can't cope with multiconfig
dependencies. The choice is either to start adding code to each of them to
filter out multiconfig dependencies, or do this at source.

After consideration we've decided to do this at source as doing otherwise
is code duplication and error prone and in any case we've looked at, they
don't make sense.

[YOCTO #13090]
[YOCTO #13130]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/runqueue.py | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 71f178de4f..329cda33a4 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -50,6 +50,11 @@ def fn_from_tid(tid):
 def taskname_from_tid(tid):
     return tid.rsplit(":", 1)[1]
 
+def mc_from_tid(tid):
+    if tid.startswith('multiconfig:'):
+        return tid.split(':')[1]
+    return ""
+
 def split_tid(tid):
     (mc, fn, taskname, _) = split_tid_mcfn(tid)
     return (mc, fn, taskname)
@@ -2102,10 +2107,23 @@ class RunQueueExecuteTasks(RunQueueExecute):
 
         return True
 
+    def filtermcdeps(self, task, deps):
+        ret = set()
+        mainmc = mc_from_tid(task)
+        for dep in deps:
+            mc = mc_from_tid(dep)
+            if mc != mainmc:
+                continue
+            ret.add(dep)
+        return ret
+
+    # We filter out multiconfig dependencies from taskdepdata we pass to the tasks 
+    # as most code can't handle them
     def build_taskdepdata(self, task):
         taskdepdata = {}
         next = self.rqdata.runtaskentries[task].depends
         next.add(task)
+        next = self.filtermcdeps(task, next)
         while next:
             additional = []
             for revdep in next:
@@ -2115,6 +2133,7 @@ class RunQueueExecuteTasks(RunQueueExecute):
                 provides = self.rqdata.dataCaches[mc].fn_provides[taskfn]
                 taskhash = self.rqdata.runtaskentries[revdep].hash
                 unihash = self.rqdata.runtaskentries[revdep].unihash
+                deps = self.filtermcdeps(task, deps)
                 taskdepdata[revdep] = [pn, taskname, fn, deps, provides, taskhash, unihash]
                 for revdep2 in deps:
                     if revdep2 not in taskdepdata:
-- 
2.20.1

[PATCH 2/3] siggen: Fix multiconfig corner case

[Richard Purdie]

There was already a fix to ignore some multiconfig dependencies but its
'opposite' case wasn't covered. Cover that combination to so as to avoid
tracebacks in multiconfig builds.

[YOCTO #13090]
[YOCTO #13130]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/siggen.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py
index 03aa08bb94..09c9c8a25f 100644
--- a/lib/bb/siggen.py
+++ b/lib/bb/siggen.py
@@ -184,6 +184,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
                 depmc = pkgname.split(':')[1]
                 if mc != depmc:
                     continue
+            if dep.startswith("multiconfig:") and not mc:
+                continue
             depname = dataCache.pkg_fn[pkgname]
             if not self.rundep_check(fn, recipename, task, dep, depname, dataCache):
                 continue
-- 
2.20.1

[PATCH 3/3] cooker: Tweak multiconfig dependency resolution

[Richard Purdie]

There were a couple of problems with the multiconfig dependency resolution:

- the "if mc" condition triggering this code wasn't correct, it needs
  to be "if more than one multiconfig" configured
- after adding providers we need to call add_unresolved again
  and rebuild mcdeps within the "while new" loop

By fixing these issues we allow various other combinations of multiconfig
builds to work which previously didn't.

[YOCTO #13090]
[YOCTO #13130]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/cooker.py | 53 ++++++++++++++++++++++--------------------------
 1 file changed, 24 insertions(+), 29 deletions(-)

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index e6b8d880ae..1982e880ba 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -641,35 +641,30 @@ class BBCooker:
 
 
         # No need to do check providers if there are no mcdeps or not an mc build
-        if mc:
-            # Add unresolved first, so we can get multiconfig indirect dependencies on time
-            for mcavailable in self.multiconfigs:
-                # The first element is empty
-                if mcavailable:
-                    taskdata[mcavailable].add_unresolved(localdata[mcavailable], self.recipecaches[mcavailable])
-
-
-            mcdeps = taskdata[mc].get_mcdepends()
-
-            if mcdeps:
-                # Make sure we can provide the multiconfig dependency
-                seen = set()
-                new = True
-                while new:
-                    new = False
-                    for mc in self.multiconfigs:
-                        for k in mcdeps:
-                            if k in seen:
-                                continue
-                            l = k.split(':')
-                            depmc = l[2]
-                            if depmc not in self.multiconfigs:
-                                bb.fatal("Multiconfig dependency %s depends on nonexistent mc configuration %s" % (k,depmc))
-                            else:
-                                logger.debug(1, "Adding providers for multiconfig dependency %s" % l[3])
-                                taskdata[depmc].add_provider(localdata[depmc], self.recipecaches[depmc], l[3])
-                                seen.add(k)
-                                new = True
+        if len(self.multiconfigs) > 1:
+            seen = set()
+            new = True
+            # Make sure we can provide the multiconfig dependency
+            while new:
+                mcdeps = set()
+                # Add unresolved first, so we can get multiconfig indirect dependencies on time
+                for mc in self.multiconfigs:
+                    taskdata[mc].add_unresolved(localdata[mc], self.recipecaches[mc])
+                    mcdeps |= set(taskdata[mc].get_mcdepends())
+                new = False
+                for mc in self.multiconfigs:
+                    for k in mcdeps:
+                        if k in seen:
+                            continue
+                        l = k.split(':')
+                        depmc = l[2]
+                        if depmc not in self.multiconfigs:
+                            bb.fatal("Multiconfig dependency %s depends on nonexistent mc configuration %s" % (k,depmc))
+                        else:
+                            logger.debug(1, "Adding providers for multiconfig dependency %s" % l[3])
+                            taskdata[depmc].add_provider(localdata[depmc], self.recipecaches[depmc], l[3])
+                            seen.add(k)
+                            new = True
 
         for mc in self.multiconfigs:
             taskdata[mc].add_unresolved(localdata[mc], self.recipecaches[mc])
-- 
2.20.1