All of lore.kernel.org
 help / color / mirror / Atom feed
* [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes
@ 2022-10-13 19:25 Jose Quaresma
  2022-10-13 19:25 ` [OE-core][PATCH 2/2] archiver: avoid using machine variable as it breaks multiconfig Jose Quaresma
  2022-10-14  9:38 ` [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Alexandre Belloni
  0 siblings, 2 replies; 4+ messages in thread
From: Jose Quaresma @ 2022-10-13 19:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: ricardo, Jose Quaresma

Test that the shared recipes in original mode with diff enabled works in multiconfig,
otherwise it will not build when using the same TMP dir.

The test can be run with:

oe-selftest -r archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch

| oe-selftest - INFO - test_archiver_multiconfig_shared_unpack_and_patch (archiver.Archiver)
| oe-selftest - INFO -  ... ok
| oe-selftest - INFO - ----------------------------------------------------------------------
| oe-selftest - INFO - Ran 1 test in 199.710s
| oe-selftest - INFO - OK
| oe-selftest - INFO - RESULTS:
| oe-selftest - INFO - RESULTS - archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch: PASSED (192.24s)
| oe-selftest - INFO - SUMMARY:
| oe-selftest - INFO - oe-selftest () - Ran 1 test in 199.711s
| oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
---
 meta/lib/oeqa/selftest/cases/archiver.py | 29 ++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/archiver.py b/meta/lib/oeqa/selftest/cases/archiver.py
index ffdea832be..1d4985dacf 100644
--- a/meta/lib/oeqa/selftest/cases/archiver.py
+++ b/meta/lib/oeqa/selftest/cases/archiver.py
@@ -6,6 +6,7 @@
 
 import os
 import glob
+import re
 from oeqa.utils.commands import bitbake, get_bb_vars
 from oeqa.selftest.case import OESelftestTestCase
 
@@ -119,7 +120,35 @@ class Archiver(OESelftestTestCase):
         excluded_present = len(glob.glob(src_path_target + '/%s-*/*' % target_recipes[1]))
         self.assertFalse(excluded_present, 'Recipe %s was not excluded.' % target_recipes[1])
 
+    def test_archiver_multiconfig_shared_unpack_and_patch(self):
+        """
+        Test that shared recipes in original mode with diff enabled works in multiconfig,
+        otherwise it will not build when using the same TMP dir.
+        """
+
+        features = 'BBMULTICONFIG = "mc1 mc2"\n'
+        features += 'INHERIT += "archiver"\n'
+        features += 'ARCHIVER_MODE[src] = "original"\n'
+        features += 'ARCHIVER_MODE[diff] = "1"\n'
+        self.write_config(features)
+
+        # We can use any machine in multiconfig as long as they are different
+        self.write_config('MACHINE = "qemuarm"\n', 'mc1')
+        self.write_config('MACHINE = "qemux86"\n', 'mc2')
+
+        task = 'do_unpack_and_patch'
+        # Use gcc-source as it is a shared recipe (appends the pv to the pn)
+        pn = 'gcc-source-%s' % get_bb_vars(['PV'], 'gcc')['PV']
+
+        # Generate the tasks signatures
+        bitbake('mc:mc1:%s mc:mc2:%s --runonly=%s --dump-signatures=none' % (pn, pn, task))
 
+        # Check the tasks signatures
+        # To be machine agnostic the tasks needs to generate the same signature for each machine
+        locked_sigs = open("%s/locked-sigs.inc" % self.builddir).read()
+        task_sigs = re.findall(r"%s:%s:.*" % (pn, task), locked_sigs)
+        uniq_sigs = set(task_sigs)
+        self.assertFalse(len(uniq_sigs) - 1, 'The task "%s" of the recipe "%s" has diferent signatures for each machine in multiconfig' % (task, pn))
 
     def test_archiver_srpm_mode(self):
         """
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [OE-core][PATCH 2/2] archiver: avoid using machine variable as it breaks multiconfig
  2022-10-13 19:25 [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Jose Quaresma
@ 2022-10-13 19:25 ` Jose Quaresma
  2022-10-14  9:38 ` [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Alexandre Belloni
  1 sibling, 0 replies; 4+ messages in thread
From: Jose Quaresma @ 2022-10-13 19:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: ricardo, Jose Quaresma

STAGING_KERNEL_DIR uses the MACHINE name so it breaks the multiconfig
and in this cases it will run the shared recipes twice, one for each
machine.

STAGING_KERNEL_DIR it's been introduced in commit 5487dee2e1

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
---
 meta/classes/archiver.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 0710c1ec5e..4049694d85 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -465,7 +465,7 @@ def is_work_shared(d):
     pn = d.getVar('PN')
     return pn.startswith('gcc-source') or \
         bb.data.inherits_class('kernel', d) or \
-        (bb.data.inherits_class('kernelsrc', d) and d.getVar('S') == d.getVar('STAGING_KERNEL_DIR'))
+        (bb.data.inherits_class('kernelsrc', d) and d.expand("${TMPDIR}/work-shared") in d.getVar('S'))
 
 # Run do_unpack and do_patch
 python do_unpack_and_patch() {
-- 
2.34.1



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes
  2022-10-13 19:25 [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Jose Quaresma
  2022-10-13 19:25 ` [OE-core][PATCH 2/2] archiver: avoid using machine variable as it breaks multiconfig Jose Quaresma
@ 2022-10-14  9:38 ` Alexandre Belloni
  2022-10-14 10:14   ` Jose Quaresma
  1 sibling, 1 reply; 4+ messages in thread
From: Alexandre Belloni @ 2022-10-14  9:38 UTC (permalink / raw)
  To: Jose Quaresma; +Cc: openembedded-core, ricardo, Jose Quaresma

Hello Jose,

This fails on the AB:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/4236/steps/15/logs/stdio

2022-10-13 21:04:28,387 - oe-selftest - INFO - archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch (subunit.RemotedTestCase)
2022-10-13 21:04:28,388 - oe-selftest - INFO -  ... FAIL
2022-10-13 21:04:28,388 - oe-selftest - INFO - 0: 15/57 149/504 (49.12s) (archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch)
2022-10-13 21:04:28,388 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/archiver.py", line 151, in test_archiver_multiconfig_shared_unpack_and_patch
    self.assertFalse(len(uniq_sigs) - 1, 'The task "%s" of the recipe "%s" has diferent signatures for each machine in multiconfig' % (task, pn))
  File "/usr/lib64/python3.6/unittest/case.py", line 693, in assertFalse
    raise self.failureException(msg)
AssertionError: -1 is not false : The task "do_unpack_and_patch" of the recipe "gcc-source-12.2.0" has diferent signatures for each machine in multiconfig

On 13/10/2022 19:25:29+0000, Jose Quaresma wrote:
> Test that the shared recipes in original mode with diff enabled works in multiconfig,
> otherwise it will not build when using the same TMP dir.
> 
> The test can be run with:
> 
> oe-selftest -r archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch
> 
> | oe-selftest - INFO - test_archiver_multiconfig_shared_unpack_and_patch (archiver.Archiver)
> | oe-selftest - INFO -  ... ok
> | oe-selftest - INFO - ----------------------------------------------------------------------
> | oe-selftest - INFO - Ran 1 test in 199.710s
> | oe-selftest - INFO - OK
> | oe-selftest - INFO - RESULTS:
> | oe-selftest - INFO - RESULTS - archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch: PASSED (192.24s)
> | oe-selftest - INFO - SUMMARY:
> | oe-selftest - INFO - oe-selftest () - Ran 1 test in 199.711s
> | oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
> 
> Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
> ---
>  meta/lib/oeqa/selftest/cases/archiver.py | 29 ++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
> 
> diff --git a/meta/lib/oeqa/selftest/cases/archiver.py b/meta/lib/oeqa/selftest/cases/archiver.py
> index ffdea832be..1d4985dacf 100644
> --- a/meta/lib/oeqa/selftest/cases/archiver.py
> +++ b/meta/lib/oeqa/selftest/cases/archiver.py
> @@ -6,6 +6,7 @@
>  
>  import os
>  import glob
> +import re
>  from oeqa.utils.commands import bitbake, get_bb_vars
>  from oeqa.selftest.case import OESelftestTestCase
>  
> @@ -119,7 +120,35 @@ class Archiver(OESelftestTestCase):
>          excluded_present = len(glob.glob(src_path_target + '/%s-*/*' % target_recipes[1]))
>          self.assertFalse(excluded_present, 'Recipe %s was not excluded.' % target_recipes[1])
>  
> +    def test_archiver_multiconfig_shared_unpack_and_patch(self):
> +        """
> +        Test that shared recipes in original mode with diff enabled works in multiconfig,
> +        otherwise it will not build when using the same TMP dir.
> +        """
> +
> +        features = 'BBMULTICONFIG = "mc1 mc2"\n'
> +        features += 'INHERIT += "archiver"\n'
> +        features += 'ARCHIVER_MODE[src] = "original"\n'
> +        features += 'ARCHIVER_MODE[diff] = "1"\n'
> +        self.write_config(features)
> +
> +        # We can use any machine in multiconfig as long as they are different
> +        self.write_config('MACHINE = "qemuarm"\n', 'mc1')
> +        self.write_config('MACHINE = "qemux86"\n', 'mc2')
> +
> +        task = 'do_unpack_and_patch'
> +        # Use gcc-source as it is a shared recipe (appends the pv to the pn)
> +        pn = 'gcc-source-%s' % get_bb_vars(['PV'], 'gcc')['PV']
> +
> +        # Generate the tasks signatures
> +        bitbake('mc:mc1:%s mc:mc2:%s --runonly=%s --dump-signatures=none' % (pn, pn, task))
>  
> +        # Check the tasks signatures
> +        # To be machine agnostic the tasks needs to generate the same signature for each machine
> +        locked_sigs = open("%s/locked-sigs.inc" % self.builddir).read()
> +        task_sigs = re.findall(r"%s:%s:.*" % (pn, task), locked_sigs)
> +        uniq_sigs = set(task_sigs)
> +        self.assertFalse(len(uniq_sigs) - 1, 'The task "%s" of the recipe "%s" has diferent signatures for each machine in multiconfig' % (task, pn))
>  
>      def test_archiver_srpm_mode(self):
>          """
> -- 
> 2.34.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#171719): https://lists.openembedded.org/g/openembedded-core/message/171719
> Mute This Topic: https://lists.openembedded.org/mt/94312275/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes
  2022-10-14  9:38 ` [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Alexandre Belloni
@ 2022-10-14 10:14   ` Jose Quaresma
  0 siblings, 0 replies; 4+ messages in thread
From: Jose Quaresma @ 2022-10-14 10:14 UTC (permalink / raw)
  To: Alexandre Belloni; +Cc: openembedded-core, ricardo, Jose Quaresma

[-- Attachment #1: Type: text/plain, Size: 5955 bytes --]

Hi Alexandre,

Alexandre Belloni <alexandre.belloni@bootlin.com> escreveu no dia sexta,
14/10/2022 à(s) 10:38:

> Hello Jose,
>
> This fails on the AB:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/4236/steps/15/logs/stdio


Oh damn, so it seems that parsing the locked-sigs.inc file is not a good
aproucth.

Does anyone know which is the best method to get the signature of the tasks
in selftest?
I need to investigate further.

Jose


>
> 2022-10-13 21:04:28,387 - oe-selftest - INFO -
> archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch
> (subunit.RemotedTestCase)
> 2022-10-13 21:04:28,388 - oe-selftest - INFO -  ... FAIL
> 2022-10-13 21:04:28,388 - oe-selftest - INFO - 0: 15/57 149/504 (49.12s)
> (archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch)
> 2022-10-13 21:04:28,388 - oe-selftest - INFO -
> testtools.testresult.real._StringException: Traceback (most recent call
> last):
>   File
> "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/archiver.py",
> line 151, in test_archiver_multiconfig_shared_unpack_and_patch
>     self.assertFalse(len(uniq_sigs) - 1, 'The task "%s" of the recipe "%s"
> has diferent signatures for each machine in multiconfig' % (task, pn))
>   File "/usr/lib64/python3.6/unittest/case.py", line 693, in assertFalse
>     raise self.failureException(msg)
> AssertionError: -1 is not false : The task "do_unpack_and_patch" of the
> recipe "gcc-source-12.2.0" has diferent signatures for each machine in
> multiconfig
>
> On 13/10/2022 19:25:29+0000, Jose Quaresma wrote:
> > Test that the shared recipes in original mode with diff enabled works in
> multiconfig,
> > otherwise it will not build when using the same TMP dir.
> >
> > The test can be run with:
> >
> > oe-selftest -r
> archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch
> >
> > | oe-selftest - INFO - test_archiver_multiconfig_shared_unpack_and_patch
> (archiver.Archiver)
> > | oe-selftest - INFO -  ... ok
> > | oe-selftest - INFO -
> ----------------------------------------------------------------------
> > | oe-selftest - INFO - Ran 1 test in 199.710s
> > | oe-selftest - INFO - OK
> > | oe-selftest - INFO - RESULTS:
> > | oe-selftest - INFO - RESULTS -
> archiver.Archiver.test_archiver_multiconfig_shared_unpack_and_patch: PASSED
> (192.24s)
> > | oe-selftest - INFO - SUMMARY:
> > | oe-selftest - INFO - oe-selftest () - Ran 1 test in 199.711s
> > | oe-selftest - INFO - oe-selftest - OK - All required tests passed
> (successes=1, skipped=0, failures=0, errors=0)
> >
> > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
> > ---
> >  meta/lib/oeqa/selftest/cases/archiver.py | 29 ++++++++++++++++++++++++
> >  1 file changed, 29 insertions(+)
> >
> > diff --git a/meta/lib/oeqa/selftest/cases/archiver.py
> b/meta/lib/oeqa/selftest/cases/archiver.py
> > index ffdea832be..1d4985dacf 100644
> > --- a/meta/lib/oeqa/selftest/cases/archiver.py
> > +++ b/meta/lib/oeqa/selftest/cases/archiver.py
> > @@ -6,6 +6,7 @@
> >
> >  import os
> >  import glob
> > +import re
> >  from oeqa.utils.commands import bitbake, get_bb_vars
> >  from oeqa.selftest.case import OESelftestTestCase
> >
> > @@ -119,7 +120,35 @@ class Archiver(OESelftestTestCase):
> >          excluded_present = len(glob.glob(src_path_target + '/%s-*/*' %
> target_recipes[1]))
> >          self.assertFalse(excluded_present, 'Recipe %s was not
> excluded.' % target_recipes[1])
> >
> > +    def test_archiver_multiconfig_shared_unpack_and_patch(self):
> > +        """
> > +        Test that shared recipes in original mode with diff enabled
> works in multiconfig,
> > +        otherwise it will not build when using the same TMP dir.
> > +        """
> > +
> > +        features = 'BBMULTICONFIG = "mc1 mc2"\n'
> > +        features += 'INHERIT += "archiver"\n'
> > +        features += 'ARCHIVER_MODE[src] = "original"\n'
> > +        features += 'ARCHIVER_MODE[diff] = "1"\n'
> > +        self.write_config(features)
> > +
> > +        # We can use any machine in multiconfig as long as they are
> different
> > +        self.write_config('MACHINE = "qemuarm"\n', 'mc1')
> > +        self.write_config('MACHINE = "qemux86"\n', 'mc2')
> > +
> > +        task = 'do_unpack_and_patch'
> > +        # Use gcc-source as it is a shared recipe (appends the pv to
> the pn)
> > +        pn = 'gcc-source-%s' % get_bb_vars(['PV'], 'gcc')['PV']
> > +
> > +        # Generate the tasks signatures
> > +        bitbake('mc:mc1:%s mc:mc2:%s --runonly=%s
> --dump-signatures=none' % (pn, pn, task))
> >
> > +        # Check the tasks signatures
> > +        # To be machine agnostic the tasks needs to generate the same
> signature for each machine
> > +        locked_sigs = open("%s/locked-sigs.inc" % self.builddir).read()
> > +        task_sigs = re.findall(r"%s:%s:.*" % (pn, task), locked_sigs)
> > +        uniq_sigs = set(task_sigs)
> > +        self.assertFalse(len(uniq_sigs) - 1, 'The task "%s" of the
> recipe "%s" has diferent signatures for each machine in multiconfig' %
> (task, pn))
> >
> >      def test_archiver_srpm_mode(self):
> >          """
> > --
> > 2.34.1
> >
>
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#171719):
> https://lists.openembedded.org/g/openembedded-core/message/171719
> > Mute This Topic: https://lists.openembedded.org/mt/94312275/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>


-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 8183 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-10-14 10:14 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-13 19:25 [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Jose Quaresma
2022-10-13 19:25 ` [OE-core][PATCH 2/2] archiver: avoid using machine variable as it breaks multiconfig Jose Quaresma
2022-10-14  9:38 ` [OE-core][PATCH 1/2] oeqa/selftest/archiver: Add multiconfig test for shared recipes Alexandre Belloni
2022-10-14 10:14   ` Jose Quaresma

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.