[PATCH] yocto-check-layer: Avoid bug when iterating and autoadding dependencies

[PATCH] yocto-check-layer: Avoid bug when iterating and autoadding dependencies

[Richard Purdie]

If iterating a layer with multiple components and auto-adding dependencies
the tests can break since layers are never removed and order isn't guaranteed
to account for that.

Fix this by resetting the layer list back to the original list each time
before auto-adding the dependencies in each case.

This fixes scanning of meta-openembedded in particular where the sublayers
may not be added in order of minimal dependency.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 scripts/yocto-check-layer | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
index b7c83c8b543..deba3cb4f8c 100755
--- a/scripts/yocto-check-layer
+++ b/scripts/yocto-check-layer
@@ -138,6 +138,9 @@ def main():
                 layer['type'] == LayerType.ERROR_BSP_DISTRO:
             continue
 
+        # Reset to a clean backup copy for each run
+        shutil.copyfile(bblayersconf + '.backup', bblayersconf)
+
         if check_bblayers(bblayersconf, layer['path'], logger):
             logger.info("%s already in %s. To capture initial signatures, layer under test should not present "
                "in BBLAYERS. Please remove %s from BBLAYERS." % (layer['name'], bblayersconf, layer['name']))
-- 
2.30.2

Re: [OE-core] [PATCH] yocto-check-layer: Avoid bug when iterating and autoadding dependencies

[Nicolas Dechesne]

[-- Attachment #1: Type: text/plain, Size: 2010 bytes --]

hey Richard,

better late than never they say ;)

On Sat, Apr 24, 2021 at 4:26 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> If iterating a layer with multiple components and auto-adding dependencies
> the tests can break since layers are never removed and order isn't
> guaranteed
> to account for that.
>
> Fix this by resetting the layer list back to the original list each time
> before auto-adding the dependencies in each case.
>
> This fixes scanning of meta-openembedded in particular where the sublayers
> may not be added in order of minimal dependency.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  scripts/yocto-check-layer | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
> index b7c83c8b543..deba3cb4f8c 100755
> --- a/scripts/yocto-check-layer
> +++ b/scripts/yocto-check-layer
> @@ -138,6 +138,9 @@ def main():
>                  layer['type'] == LayerType.ERROR_BSP_DISTRO:
>              continue
>
> +        # Reset to a clean backup copy for each run
> +        shutil.copyfile(bblayersconf + '.backup', bblayersconf)
> +
>

I was spending some time in yocto-check-layer, and noticed this patch. You
have the exact same statement 10 lines later. I think this patch should
have moved the statement before the call to check_bblayers() instead of
duplicating it. And I think this patch is a fix for a problem introduced in
be02e8dbfb0d (checklayer: check layer in BBLAYERS before test).

I can send a patch to remove the duplicate statement, or if simpler for
you, you can do it directly.


>          if check_bblayers(bblayersconf, layer['path'], logger):
>              logger.info("%s already in %s. To capture initial
> signatures, layer under test should not present "
>                 "in BBLAYERS. Please remove %s from BBLAYERS." %
> (layer['name'], bblayersconf, layer['name']))
> --
> 2.30.2
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 2852 bytes --]

Re: [OE-core] [PATCH] yocto-check-layer: Avoid bug when iterating and autoadding dependencies

[Richard Purdie]

On Thu, 2021-07-22 at 11:34 +0200, Nicolas Dechesne wrote:
> hey Richard,
> 
> better late than never they say ;) 
> 
> On Sat, Apr 24, 2021 at 4:26 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> > If iterating a layer with multiple components and auto-adding dependencies
> > the tests can break since layers are never removed and order isn't guaranteed
> > to account for that.
> > 
> > Fix this by resetting the layer list back to the original list each time
> > before auto-adding the dependencies in each case.
> > 
> > This fixes scanning of meta-openembedded in particular where the sublayers
> > may not be added in order of minimal dependency.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> >  scripts/yocto-check-layer | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
> > index b7c83c8b543..deba3cb4f8c 100755
> > --- a/scripts/yocto-check-layer
> > +++ b/scripts/yocto-check-layer
> > @@ -138,6 +138,9 @@ def main():
> >                  layer['type'] == LayerType.ERROR_BSP_DISTRO:
> >              continue
> > 
> > +        # Reset to a clean backup copy for each run
> > +        shutil.copyfile(bblayersconf + '.backup', bblayersconf)
> > +
> > 
> 
> 
> I was spending some time in yocto-check-layer, and noticed this patch. You have 
> the exact same statement 10 lines later. I think this patch should have moved the 
> statement before the call to check_bblayers() instead of duplicating it. And I 
> think this patch is a fix for a problem introduced in be02e8dbfb0d (checklayer: 
> check layer in BBLAYERS before test).
> 
> I can send a patch to remove the duplicate statement, or if simpler for you, you 
> can do it directly. 

I've queued and sent out a patch, well spotted, thanks!

Cheers,

Richard