[PATCH] runqueue: Ensure only recursive task dependencies are pruned

[PATCH] runqueue: Ensure only recursive task dependencies are pruned

[Richard Purdie]

If a standalone tasks adds a dependency on X:do_build, the code in runqueue would
currently remove it if that do_build was part of an image recipe which uses
recrdeptask on do_build.

Such individual tasks shouldn't do this, therefore tweak the recursive reference code
to only process recurseive tasks, not all tasks.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/runqueue.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 48df013..f2e52cf 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -765,7 +765,7 @@ class RunQueueData:
             bb.debug(1, "Added %s recursive dependencies in this loop" % extradeps)
 
         # Remove recrdeptask circular references so that do_a[recrdeptask] = "do_a do_b" can work
-        for tid in self.runtaskentries:
+        for tid in recursivetasksselfref:
             self.runtaskentries[tid].depends.difference_update(recursivetasksselfref)
 
         self.init_progress_reporter.next_stage()
-- 
2.7.4

Re: [PATCH] runqueue: Ensure only recursive task dependencies are pruned

[Richard Purdie]

On Tue, 2018-02-27 at 21:48 +0000, Richard Purdie wrote:
> If a standalone tasks adds a dependency on X:do_build, the code in
> runqueue would
> currently remove it if that do_build was part of an image recipe
> which uses
> recrdeptask on do_build.
> 
> Such individual tasks shouldn't do this, therefore tweak the
> recursive reference code
> to only process recurseive tasks, not all tasks.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  lib/bb/runqueue.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
> index 48df013..f2e52cf 100644
> --- a/lib/bb/runqueue.py
> +++ b/lib/bb/runqueue.py
> @@ -765,7 +765,7 @@ class RunQueueData:
>              bb.debug(1, "Added %s recursive dependencies in this
> loop" % extradeps)
>  
>          # Remove recrdeptask circular references so that
> do_a[recrdeptask] = "do_a do_b" can work
> -        for tid in self.runtaskentries:
> +        for tid in recursivetasksselfref:
>              self.runtaskentries[tid].depends.difference_update(recur
> sivetasksselfref)
>  
>          self.init_progress_reporter.next_stage()

We have a problem with this patch since it causes:

$ oe-selftest -r sstatetests.SStateTests.test_sstate_sametune_samesigs
2018-02-28 10:33:56,968 - oe-selftest - INFO - Adding layer libraries:
2018-02-28 10:33:56,968 - oe-selftest - INFO - 	/media/build1/poky/meta/lib
2018-02-28 10:33:56,969 - oe-selftest - INFO - 	/media/build1/poky/meta-yocto-bsp/lib
2018-02-28 10:33:56,969 - oe-selftest - INFO - 	/media/build1/poky/meta-selftest/lib
2018-02-28 10:33:56,969 - oe-selftest - INFO - Running bitbake -p
2018-02-28 10:34:02,651 - oe-selftest - INFO - Adding: "include selftest.inc" in /media/build1/poky/build/conf/local.conf
2018-02-28 10:34:02,652 - oe-selftest - INFO - test_sstate_sametune_samesigs (sstatetests.SStateTests)
2018-02-28 10:36:13,404 - oe-selftest - INFO -  ... FAIL
2018-02-28 10:36:13,406 - oe-selftest - INFO - ======================================================================
2018-02-28 10:36:13,406 - oe-selftest - INFO - FAIL: test_sstate_sametune_samesigs (sstatetests.SStateTests)
2018-02-28 10:36:13,406 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-02-28 10:36:13,406 - oe-selftest - INFO - Traceback (most recent call last):
  File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/media/build1/poky/meta/lib/oeqa/selftest/cases/sstatetests.py", line 449, in test_sstate_sametune_samesigs
    self.assertCountEqual(files1, files2)
AssertionError: Element counts were not equal:
First has 1, Second has 0:  '/media/build1/poky/build/tmp-sstatesamehash/stamps/i586-poky-linux/meta-world-pkgdata/1.0-r0.do_allpackagedata.sigdata.d002ff3e63a32eb99a2cec0db32958ea'
First has 1, Second has 0:  '/media/build1/poky/build/tmp-sstatesamehash/stamps/x86-pokymllib32-linux/lib32-meta-world-pkgdata/1.0-r0.do_allpackagedata.sigdata.3408a479c43dc528f7087d1f49d048a1'
First has 0, Second has 1:  '/media/build1/poky/build/tmp-sstatesamehash/stamps/i586-poky-linux/meta-world-pkgdata/1.0-r0.do_allpackagedata.sigdata.870804331fb653c926de0e5837f4634a'
First has 0, Second has 1:  '/media/build1/poky/build/tmp-sstatesamehash/stamps/x86-pokymllib32-linux/lib32-meta-world-pkgdata/1.0-r0.do_allpackagedata.sigdata.e99b43d58e820c369e8d7e717bac41a2'

Cheers,

Richard

Re: [PATCH] runqueue: Ensure only recursive task dependencies are pruned

[Richard Purdie]

On Wed, 2018-02-28 at 10:44 +0000, Richard Purdie wrote:
> We have a problem with this patch since it causes:
> 
> $ oe-selftest -r
> sstatetests.SStateTests.test_sstate_sametune_samesigs
[...]
> =====================================================================
> =
> 2018-02-28 10:36:13,406 - oe-selftest - INFO - FAIL:
> test_sstate_sametune_samesigs (sstatetests.SStateTests)
> 2018-02-28 10:36:13,406 - oe-selftest - INFO - ----------------------

Further investigation suggests this is a metadata bug and bitbake is
right here. Working on a fix for the metadata.

Cheers,

Richard