[PATCH 0/4] yocto-check-layer: add support to check for dependencies

[PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Nicolas Dechesne]

It has been recentely mentioned to me by the YP TSC that a layer must
not only pass the yocto-check-layer script, but also all its
dependencies. These patches add support for testing dependencies
automatically.

Patch #1 and #3 are not much related, but improvements I made along
the lines. 

Patch #2 split the internal function into 2 parts, so that we can get
the list of dependencies from the main script. 

Patch #4 is where the processing of the dependencies is done. I chose
to enable checking the dependencies by default, and use
'--no-auto-dependency' to disable it, we could decide the other way
around. For any dependency of the 'layers under test' we will simply
add them to the list of layers to test unless they are already there.

Here is a sample output with these changes:

$ yocto-check-layer /work/oe/sources/meta-aws \ 
                    --dependency /work/oe/sources/meta-openembedded/ 

INFO: Summary of results:
INFO:
INFO: meta-aws ... PASS
INFO: meta-python ... PASS
INFO: meta-oe ... PASS
INFO: meta-networking ... PASS

Nicolas Dechesne (4):
  yocto-check-layer: improve missed dependencies
  checklayer: new function get_layer_dependencies()
  checklayer: rename _find_layer_depends
  yocto-check-layer: ensure that all layer dependencies are tested too

 scripts/lib/checklayer/__init__.py | 11 ++++++++---
 scripts/yocto-check-layer          | 23 +++++++++++++++++++----
 2 files changed, 27 insertions(+), 7 deletions(-)

-- 
2.29.2

[PATCH 1/4] yocto-check-layer: improve missed dependencies

[Nicolas Dechesne]

The first 2 calls to add_layer_dependencies() are here to add all
dependencies for the 'layer under test' and the additional layers
provided by the users.

In both cases, we use misssing_dependencies boolean to indicate if any
dependency is missing. But we then never really use
missing_dependencies. Instead the script is calling
add_layer_dependencies() again (for both the layer under test, and the
additional layers) to detect if there are any missing dependency. As a
result, we are trying to add again all dependencies, and we can see
that from the traces:

INFO: Detected layers:
INFO: meta-aws: LayerType.SOFTWARE, /work/oe/sources/meta-aws
INFO: checklayer: Doesn't have conf/layer.conf file, so ignoring
INFO:
INFO: Setting up for meta-aws(LayerType.SOFTWARE), /work/oe/sources/meta-aws
INFO: Adding layer meta-python
INFO: Adding layer meta-oe
INFO: Adding layer meta-networking
-->
INFO: Adding layer meta-python
INFO: meta-python is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
INFO: Adding layer meta-oe
INFO: meta-oe is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
INFO: Adding layer meta-networking
INFO: meta-networking is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
<--
INFO: Getting initial bitbake variables ...

The code appears more complex than it should, and we can simply
replace the complex if statement by using missing_dependencies, and
avoid duplicating the call to add_layer_dependencies().

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
---
 scripts/yocto-check-layer | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
index 44e77b73d0..a179240fd0 100755
--- a/scripts/yocto-check-layer
+++ b/scripts/yocto-check-layer
@@ -160,9 +160,7 @@ def main():
                 if not add_layer_dependencies(bblayersconf, additional_layer, dep_layers, logger):
                     missing_dependencies = True
                     break
-        if not add_layer_dependencies(bblayersconf, layer, dep_layers, logger) or \
-           any(map(lambda additional_layer: not add_layer_dependencies(bblayersconf, additional_layer, dep_layers, logger),
-                   additional_layers)):
+        if missing_dependencies:
             logger.info('Skipping %s due to missing dependencies.' % layer['name'])
             results[layer['name']] = None
             results_status[layer['name']] = 'SKIPPED (Missing dependencies)'
-- 
2.29.2

[PATCH 2/4] checklayer: new function get_layer_dependencies()

[Nicolas Dechesne]

Split add_layer_dependencies() into 2 parts. First search for layer
dependencies, and then add them to the config. That allows us to
call get_layer_dependencies() independently.

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
---
 scripts/lib/checklayer/__init__.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/checklayer/__init__.py b/scripts/lib/checklayer/__init__.py
index fe545607bb..72d9df0a62 100644
--- a/scripts/lib/checklayer/__init__.py
+++ b/scripts/lib/checklayer/__init__.py
@@ -156,7 +156,7 @@ def _find_layer_depends(depend, layers):
                 return layer
     return None
 
-def add_layer_dependencies(bblayersconf, layer, layers, logger):
+def get_layer_dependencies(layer, layers, logger):
     def recurse_dependencies(depends, layer, layers, logger, ret = []):
         logger.debug('Processing dependencies %s for layer %s.' % \
                     (depends, layer['name']))
@@ -203,6 +203,11 @@ def add_layer_dependencies(bblayersconf, layer, layers, logger):
         layer_depends = recurse_dependencies(depends, layer, layers, logger, layer_depends)
 
     # Note: [] (empty) is allowed, None is not!
+    return layer_depends
+
+def add_layer_dependencies(bblayersconf, layer, layers, logger):
+
+    layer_depends = get_layer_dependencies(layer, layers, logger)
     if layer_depends is None:
         return False
     else:
-- 
2.29.2

[PATCH 3/4] checklayer: rename _find_layer_depends

[Nicolas Dechesne]

What this function does is really to find a layer, not a 'depends'. We
are using this function to find a dependent layer, but the name is
confusing.

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
---
 scripts/lib/checklayer/__init__.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/checklayer/__init__.py b/scripts/lib/checklayer/__init__.py
index 72d9df0a62..e69a10f452 100644
--- a/scripts/lib/checklayer/__init__.py
+++ b/scripts/lib/checklayer/__init__.py
@@ -146,7 +146,7 @@ def detect_layers(layer_directories, no_auto):
 
     return layers
 
-def _find_layer_depends(depend, layers):
+def _find_layer(depend, layers):
     for layer in layers:
         if 'collections' not in layer:
             continue
@@ -166,7 +166,7 @@ def get_layer_dependencies(layer, layers, logger):
             if depend == 'core':
                 continue
 
-            layer_depend = _find_layer_depends(depend, layers)
+            layer_depend = _find_layer(depend, layers)
             if not layer_depend:
                 logger.error('Layer %s depends on %s and isn\'t found.' % \
                         (layer['name'], depend))
-- 
2.29.2

[PATCH 4/4] yocto-check-layer: ensure that all layer dependencies are tested too

[Nicolas Dechesne]

In order to be compliant with the YP compatible status, a layer also
needs to ensure that all its dependencies are compatible
too. Currently yocto-check-layer only checks the requested layer,
without testing any dependencies.

With this change, all dependencies are also checked by default, so the
summary printed at the end will give a clear picture whether all
dependencies pass the script or not.

Using --no-auto-dependency can be used to skip that.

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
---
 scripts/yocto-check-layer | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
index a179240fd0..e3a4c940ec 100755
--- a/scripts/yocto-check-layer
+++ b/scripts/yocto-check-layer
@@ -24,7 +24,7 @@ import scriptpath
 scriptpath.add_oe_lib_path()
 scriptpath.add_bitbake_lib_path()
 
-from checklayer import LayerType, detect_layers, add_layers, add_layer_dependencies, get_signatures, check_bblayers
+from checklayer import LayerType, detect_layers, add_layers, add_layer_dependencies, get_layer_dependencies, get_signatures, check_bblayers
 from oeqa.utils.commands import get_bb_vars
 
 PROGNAME = 'yocto-check-layer'
@@ -51,6 +51,8 @@ def main():
             help='File to output log (optional)', action='store')
     parser.add_argument('--dependency', nargs="+",
             help='Layers to process for dependencies', action='store')
+    parser.add_argument('--no-auto-dependency', help='Disable automatic testing of dependencies',
+            action='store_true')
     parser.add_argument('--machines', nargs="+",
             help='List of MACHINEs to be used during testing', action='store')
     parser.add_argument('--additional-layers', nargs="+",
@@ -121,6 +123,21 @@ def main():
     if not layers:
         return 1
 
+    # Find all dependencies, and get them checked too
+    if not args.no_auto_dependency:
+        depends = []
+        for layer in layers:
+            layer_depends = get_layer_dependencies(layer, dep_layers, logger)
+            if layer_depends:
+                for d in layer_depends:
+                    if d not in depends:
+                        depends.append(d)
+
+        for d in depends:
+            if d not in layers:
+                logger.info("Adding %s to the list of layers to test, as a dependency", d['name'])
+                layers.append(d)
+
     shutil.copyfile(bblayersconf, bblayersconf + '.backup')
     def cleanup_bblayers(signum, frame):
         shutil.copyfile(bblayersconf + '.backup', bblayersconf)
-- 
2.29.2

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Richard Purdie]

On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
> It has been recentely mentioned to me by the YP TSC that a layer must
> not only pass the yocto-check-layer script, but also all its
> dependencies. These patches add support for testing dependencies
> automatically.
> 
> Patch #1 and #3 are not much related, but improvements I made along
> the lines. 
> 
> Patch #2 split the internal function into 2 parts, so that we can get
> the list of dependencies from the main script. 
> 
> Patch #4 is where the processing of the dependencies is done. I chose
> to enable checking the dependencies by default, and use
> '--no-auto-dependency' to disable it, we could decide the other way
> around. For any dependency of the 'layers under test' we will simply
> add them to the list of layers to test unless they are already there.

I'm fine with the changes, thanks for doing it. I think we'll need to add
this option to the yocto-autobuilder-helper config.json entries else
the current testing will take much longer. We're safe there to
assume that dependencies are tested separately.

Cheers,

Richard

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Nicolas Dechesne]

[-- Attachment #1: Type: text/plain, Size: 1504 bytes --]

On Thu, Jul 22, 2021 at 2:51 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
> > It has been recentely mentioned to me by the YP TSC that a layer must
> > not only pass the yocto-check-layer script, but also all its
> > dependencies. These patches add support for testing dependencies
> > automatically.
> >
> > Patch #1 and #3 are not much related, but improvements I made along
> > the lines.
> >
> > Patch #2 split the internal function into 2 parts, so that we can get
> > the list of dependencies from the main script.
> >
> > Patch #4 is where the processing of the dependencies is done. I chose
> > to enable checking the dependencies by default, and use
> > '--no-auto-dependency' to disable it, we could decide the other way
> > around. For any dependency of the 'layers under test' we will simply
> > add them to the list of layers to test unless they are already there.
>
> I'm fine with the changes, thanks for doing it. I think we'll need to add
> this option to the yocto-autobuilder-helper config.json entries else
> the current testing will take much longer. We're safe there to
> assume that dependencies are tested separately.
>

Right. This is a good point. Let's review the patches first and make sure
they work well! I really prefer if it's enabled by default, so that it's
more obvious to layers' maintainers if/when there is a problem.


>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 2135 bytes --]

Re: [OE-core] [PATCH 4/4] yocto-check-layer: ensure that all layer dependencies are tested too

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 3604 bytes --]

On Thu, Jul 22, 2021 at 8:47 AM Nicolas Dechesne <
nicolas.dechesne@linaro.org> wrote:

> In order to be compliant with the YP compatible status, a layer also
> needs to ensure that all its dependencies are compatible
> too. Currently yocto-check-layer only checks the requested layer,
> without testing any dependencies.
>

Is that actually written into our compliance statements ? (that dependency
layers must also be compliant)

I had never heard that before, and in my opinion, that will actively
encourage people to copy recipes if they want to be compliant but a
dependent layer is problematic.


>
> With this change, all dependencies are also checked by default, so the
> summary printed at the end will give a clear picture whether all
> dependencies pass the script or not.
>
Using --no-auto-dependency can be used to skip that.
>
>
I'd actually prefer the opposite, to make the compliance runs faster by
default, versus someone having to find out about this option later. We
already get complaints about check layer speed, and doing more by default
won't help on that front.

Bruce



> Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
> ---
>  scripts/yocto-check-layer | 19 ++++++++++++++++++-
>  1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
> index a179240fd0..e3a4c940ec 100755
> --- a/scripts/yocto-check-layer
> +++ b/scripts/yocto-check-layer
> @@ -24,7 +24,7 @@ import scriptpath
>  scriptpath.add_oe_lib_path()
>  scriptpath.add_bitbake_lib_path()
>
> -from checklayer import LayerType, detect_layers, add_layers,
> add_layer_dependencies, get_signatures, check_bblayers
> +from checklayer import LayerType, detect_layers, add_layers,
> add_layer_dependencies, get_layer_dependencies, get_signatures,
> check_bblayers
>  from oeqa.utils.commands import get_bb_vars
>
>  PROGNAME = 'yocto-check-layer'
> @@ -51,6 +51,8 @@ def main():
>              help='File to output log (optional)', action='store')
>      parser.add_argument('--dependency', nargs="+",
>              help='Layers to process for dependencies', action='store')
> +    parser.add_argument('--no-auto-dependency', help='Disable automatic
> testing of dependencies',
> +            action='store_true')
>      parser.add_argument('--machines', nargs="+",
>              help='List of MACHINEs to be used during testing',
> action='store')
>      parser.add_argument('--additional-layers', nargs="+",
> @@ -121,6 +123,21 @@ def main():
>      if not layers:
>          return 1
>
> +    # Find all dependencies, and get them checked too
> +    if not args.no_auto_dependency:
> +        depends = []
> +        for layer in layers:
> +            layer_depends = get_layer_dependencies(layer, dep_layers,
> logger)
> +            if layer_depends:
> +                for d in layer_depends:
> +                    if d not in depends:
> +                        depends.append(d)
> +
> +        for d in depends:
> +            if d not in layers:
> +                logger.info("Adding %s to the list of layers to test, as
> a dependency", d['name'])
> +                layers.append(d)
> +
>      shutil.copyfile(bblayersconf, bblayersconf + '.backup')
>      def cleanup_bblayers(signum, frame):
>          shutil.copyfile(bblayersconf + '.backup', bblayersconf)
> --
> 2.29.2
>
>
> 
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 5433 bytes --]

Re: [OE-core] [PATCH 4/4] yocto-check-layer: ensure that all layer dependencies are tested too

[Armin Kuster]

On 7/22/21 6:16 AM, Bruce Ashfield wrote:
>
>
> On Thu, Jul 22, 2021 at 8:47 AM Nicolas Dechesne
> <nicolas.dechesne@linaro.org <mailto:nicolas.dechesne@linaro.org>> wrote:
>
>     In order to be compliant with the YP compatible status, a layer also
>     needs to ensure that all its dependencies are compatible
>     too. Currently yocto-check-layer only checks the requested layer,
>     without testing any dependencies.
>
>
> Is that actually written into our compliance statements ? (that
> dependency layers must also be compliant)
>
> I had never heard that before, and in my opinion, that will actively
> encourage people to copy recipes if they want to be compliant but a
> dependent layer is problematic.

Yes it is. 

-Armin
>  
>
>
>     With this change, all dependencies are also checked by default, so the
>     summary printed at the end will give a clear picture whether all
>     dependencies pass the script or not. 
>
>     Using --no-auto-dependency can be used to skip that.
>
>
> I'd actually prefer the opposite, to make the compliance runs faster
> by default, versus someone having to find out about this option later.
> We already get complaints about check layer speed, and doing more by
> default won't help on that front.
>
> Bruce
>
>  
>
>     Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org
>     <mailto:nicolas.dechesne@linaro.org>>
>     ---
>      scripts/yocto-check-layer | 19 ++++++++++++++++++-
>      1 file changed, 18 insertions(+), 1 deletion(-)
>
>     diff --git a/scripts/yocto-check-layer b/scripts/yocto-check-layer
>     index a179240fd0..e3a4c940ec 100755
>     --- a/scripts/yocto-check-layer
>     +++ b/scripts/yocto-check-layer
>     @@ -24,7 +24,7 @@ import scriptpath
>      scriptpath.add_oe_lib_path()
>      scriptpath.add_bitbake_lib_path()
>
>     -from checklayer import LayerType, detect_layers, add_layers,
>     add_layer_dependencies, get_signatures, check_bblayers
>     +from checklayer import LayerType, detect_layers, add_layers,
>     add_layer_dependencies, get_layer_dependencies, get_signatures,
>     check_bblayers
>      from oeqa.utils.commands import get_bb_vars
>
>      PROGNAME = 'yocto-check-layer'
>     @@ -51,6 +51,8 @@ def main():
>                  help='File to output log (optional)', action='store')
>          parser.add_argument('--dependency', nargs="+",
>                  help='Layers to process for dependencies',
>     action='store')
>     +    parser.add_argument('--no-auto-dependency', help='Disable
>     automatic testing of dependencies',
>     +            action='store_true')
>          parser.add_argument('--machines', nargs="+",
>                  help='List of MACHINEs to be used during testing',
>     action='store')
>          parser.add_argument('--additional-layers', nargs="+",
>     @@ -121,6 +123,21 @@ def main():
>          if not layers:
>              return 1
>
>     +    # Find all dependencies, and get them checked too
>     +    if not args.no_auto_dependency:
>     +        depends = []
>     +        for layer in layers:
>     +            layer_depends = get_layer_dependencies(layer,
>     dep_layers, logger)
>     +            if layer_depends:
>     +                for d in layer_depends:
>     +                    if d not in depends:
>     +                        depends.append(d)
>     +
>     +        for d in depends:
>     +            if d not in layers:
>     +                logger.info <http://logger.info>("Adding %s to
>     the list of layers to test, as a dependency", d['name'])
>     +                layers.append(d)
>     +
>          shutil.copyfile(bblayersconf, bblayersconf + '.backup')
>          def cleanup_bblayers(signum, frame):
>              shutil.copyfile(bblayersconf + '.backup', bblayersconf)
>     -- 
>     2.29.2
>
>
>
>
>
>
> -- 
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>
> 
>

Re: [OE-core] [PATCH 4/4] yocto-check-layer: ensure that all layer dependencies are tested too

[Richard Purdie]

On Thu, 2021-07-22 at 09:16 -0400, Bruce Ashfield wrote:
> 
> 
> On Thu, Jul 22, 2021 at 8:47 AM Nicolas Dechesne <nicolas.dechesne@linaro.org> wrote:
> > In order to be compliant with the YP compatible status, a layer also
> > needs to ensure that all its dependencies are compatible
> > too. Currently yocto-check-layer only checks the requested layer,
> > without testing any dependencies.
> > 
> 
> 
> Is that actually written into our compliance statements ? (that dependency 
> layers must also be compliant)
> 
> I had never heard that before, and in my opinion, that will actively encourage 
> people to copy recipes if they want to be compliant but a dependent layer is 
> problematic.

It has been the case as it logically doesn't make sense otherwise but the 
YP Compat pieces need better documentation. The TSC did work on fixing this
up but we're blocked on a lack of advocacy people to help with the other 
side of the programme.

Thinking the above though, the reasoning is that if we don't require that, 
it lets anyone just push their non-compliant bits into meta-non-compatible 
and then have meta-compatible depend on meta-non-compatible. It also meant
nobody had much interest in having meta-oe or meta-virt being YP Compat.

The hope is that it causes "bad" layers to get fixed. We've put pieces in place
to try and at least ensure the core layers pass and stay passing.

>  With this change, all dependencies are also checked by default, so the
> > summary printed at the end will give a clear picture whether all
> > dependencies pass the script or not. 
> > 
> > Using --no-auto-dependency can be used to skip that.
> > 
> 
> 
> I'd actually prefer the opposite, to make the compliance runs faster by 
> default, versus someone having to find out about this option later. We 
> already get complaints about check layer speed, and doing more by default 
> won't help on that front.

I can see the arguments both ways on this. "The script says it is compatible
so I can use the badge" :/

Cheers,

Richard

Re: [OE-core] [PATCH 4/4] yocto-check-layer: ensure that all layer dependencies are tested too

[Bruce Ashfield]

On Thu, Jul 22, 2021 at 4:46 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2021-07-22 at 09:16 -0400, Bruce Ashfield wrote:
> >
> >
> > On Thu, Jul 22, 2021 at 8:47 AM Nicolas Dechesne <nicolas.dechesne@linaro.org> wrote:
> > > In order to be compliant with the YP compatible status, a layer also
> > > needs to ensure that all its dependencies are compatible
> > > too. Currently yocto-check-layer only checks the requested layer,
> > > without testing any dependencies.
> > >
> >
> >
> > Is that actually written into our compliance statements ? (that dependency
> > layers must also be compliant)
> >
> > I had never heard that before, and in my opinion, that will actively encourage
> > people to copy recipes if they want to be compliant but a dependent layer is
> > problematic.
>
> It has been the case as it logically doesn't make sense otherwise but the
> YP Compat pieces need better documentation. The TSC did work on fixing this
> up but we're blocked on a lack of advocacy people to help with the other
> side of the programme.
>
> Thinking the above though, the reasoning is that if we don't require that,
> it lets anyone just push their non-compliant bits into meta-non-compatible
> and then have meta-compatible depend on meta-non-compatible. It also meant
> nobody had much interest in having meta-oe or meta-virt being YP Compat.

Indeed. I can see that happening as well.

I'll cross the bridge if meta-virt goes incompatible due to a
dependency when it happens!

>
> The hope is that it causes "bad" layers to get fixed. We've put pieces in place
> to try and at least ensure the core layers pass and stay passing.
>
> >  With this change, all dependencies are also checked by default, so the
> > > summary printed at the end will give a clear picture whether all
> > > dependencies pass the script or not.
> > >
> > > Using --no-auto-dependency can be used to skip that.
> > >
> >
> >
> > I'd actually prefer the opposite, to make the compliance runs faster by
> > default, versus someone having to find out about this option later. We
> > already get complaints about check layer speed, and doing more by default
> > won't help on that front.
>
> I can see the arguments both ways on this. "The script says it is compatible
> so I can use the badge" :/

Agreed. It is obviously up to you on this. I just wanted to make sure.

Bruce

>
> Cheers,
>
> Richard
>
>
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Richard Purdie]

On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
> It has been recentely mentioned to me by the YP TSC that a layer must
> not only pass the yocto-check-layer script, but also all its
> dependencies. These patches add support for testing dependencies
> automatically.
> 
> Patch #1 and #3 are not much related, but improvements I made along
> the lines. 
> 
> Patch #2 split the internal function into 2 parts, so that we can get
> the list of dependencies from the main script. 
> 
> Patch #4 is where the processing of the dependencies is done. I chose
> to enable checking the dependencies by default, and use
> '--no-auto-dependency' to disable it, we could decide the other way
> around. For any dependency of the 'layers under test' we will simply
> add them to the list of layers to test unless they are already there.
> 
> Here is a sample output with these changes:
> 
> $ yocto-check-layer /work/oe/sources/meta-aws \ 
>                     --dependency /work/oe/sources/meta-openembedded/ 
> 
> INFO: Summary of results:
> INFO:
> INFO: meta-aws ... PASS
> INFO: meta-python ... PASS
> INFO: meta-oe ... PASS
> INFO: meta-networking ... PASS

We merged this but have started seeing:

https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/166

which seems a little strange and is probably related to this series.

Cheers,

Richard

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Nicolas Dechesne]

[-- Attachment #1: Type: text/plain, Size: 2181 bytes --]

On Fri, Jul 30, 2021 at 10:03 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
> > It has been recentely mentioned to me by the YP TSC that a layer must
> > not only pass the yocto-check-layer script, but also all its
> > dependencies. These patches add support for testing dependencies
> > automatically.
> >
> > Patch #1 and #3 are not much related, but improvements I made along
> > the lines.
> >
> > Patch #2 split the internal function into 2 parts, so that we can get
> > the list of dependencies from the main script.
> >
> > Patch #4 is where the processing of the dependencies is done. I chose
> > to enable checking the dependencies by default, and use
> > '--no-auto-dependency' to disable it, we could decide the other way
> > around. For any dependency of the 'layers under test' we will simply
> > add them to the list of layers to test unless they are already there.
> >
> > Here is a sample output with these changes:
> >
> > $ yocto-check-layer /work/oe/sources/meta-aws \
> >                     --dependency /work/oe/sources/meta-openembedded/
> >
> > INFO: Summary of results:
> > INFO:
> > INFO: meta-aws ... PASS
> > INFO: meta-python ... PASS
> > INFO: meta-oe ... PASS
> > INFO: meta-networking ... PASS
>
> We merged this but have started seeing:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/166
>
> which seems a little strange and is probably related to this series.
>

yes, it's related.

...
INFO: Adding meta-poky to the list of layers to test, as a dependency
...
INFO: meta-poky ... SKIPPED (Layer under test should not present in
BBLAYERS)

With these changes, we are now adding all dependencies and trying to test
them too. But meta-poky seems to be in bblayers.conf, and yocto-check-layer
won't work since it needs to compute all checksum *without* the layer in
the config first.

It is probably a good idea to make sure bblayers only has oe-core when we
try to perform YP compat. Alternatively, as discussed, we
use --no-auto-dependency on AB..


>
> Cheers,
>
> Richard
>
>
>

[-- Attachment #2: Type: text/html, Size: 3171 bytes --]

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Richard Purdie]

On Fri, 2021-07-30 at 11:08 +0200, Nicolas Dechesne wrote:
> 
> 
> On Fri, Jul 30, 2021 at 10:03 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> > On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
> > > It has been recentely mentioned to me by the YP TSC that a layer must
> > > not only pass the yocto-check-layer script, but also all its
> > > dependencies. These patches add support for testing dependencies
> > > automatically.
> > > 
> > > Patch #1 and #3 are not much related, but improvements I made along
> > > the lines. 
> > > 
> > > Patch #2 split the internal function into 2 parts, so that we can get
> > > the list of dependencies from the main script. 
> > > 
> > > Patch #4 is where the processing of the dependencies is done. I chose
> > > to enable checking the dependencies by default, and use
> > > '--no-auto-dependency' to disable it, we could decide the other way
> > > around. For any dependency of the 'layers under test' we will simply
> > > add them to the list of layers to test unless they are already there.
> > > 
> > > Here is a sample output with these changes:
> > > 
> > > $ yocto-check-layer /work/oe/sources/meta-aws \ 
> > >                     --dependency /work/oe/sources/meta-openembedded/ 
> > > 
> > > INFO: Summary of results:
> > > INFO:
> > > INFO: meta-aws ... PASS
> > > INFO: meta-python ... PASS
> > > INFO: meta-oe ... PASS
> > > INFO: meta-networking ... PASS
> > 
> > We merged this but have started seeing:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/166
> > 
> > which seems a little strange and is probably related to this series.
> > 
> 
> 
> yes, it's related.
> 
> ...
> INFO: Adding meta-poky to the list of layers to test, as a dependency
> ...
> INFO: meta-poky ... SKIPPED (Layer under test should not present in BBLAYERS)
> 
> With these changes, we are now adding all dependencies and trying to test them too. But meta-poky seems to
> be in bblayers.conf, and yocto-check-layer won't work since it needs to compute all checksum *without* the
> layer in the config first. 
> 
> It is probably a good idea to make sure bblayers only has oe-core when we try to perform YP compat.
> Alternatively, as discussed, we use --no-auto-dependency on AB.. 

Fair enough, I'll update the config to do that.

Cheers,

Richard

> 

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Nicolas Dechesne]

[-- Attachment #1: Type: text/plain, Size: 2858 bytes --]

Richard, Steve, Anuj,


On Fri, Jul 30, 2021 at 11:28 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2021-07-30 at 11:08 +0200, Nicolas Dechesne wrote:
> >
> >
> > On Fri, Jul 30, 2021 at 10:03 AM Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
> > > On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
> > > > It has been recentely mentioned to me by the YP TSC that a layer must
> > > > not only pass the yocto-check-layer script, but also all its
> > > > dependencies. These patches add support for testing dependencies
> > > > automatically.
> > > >
> > > > Patch #1 and #3 are not much related, but improvements I made along
> > > > the lines.
> > > >
> > > > Patch #2 split the internal function into 2 parts, so that we can get
> > > > the list of dependencies from the main script.
> > > >
> > > > Patch #4 is where the processing of the dependencies is done. I chose
> > > > to enable checking the dependencies by default, and use
> > > > '--no-auto-dependency' to disable it, we could decide the other way
> > > > around. For any dependency of the 'layers under test' we will simply
> > > > add them to the list of layers to test unless they are already there.
> > > >
> > > > Here is a sample output with these changes:
> > > >
> > > > $ yocto-check-layer /work/oe/sources/meta-aws \
> > > >                     --dependency /work/oe/sources/meta-openembedded/
> > > >
> > > > INFO: Summary of results:
> > > > INFO:
> > > > INFO: meta-aws ... PASS
> > > > INFO: meta-python ... PASS
> > > > INFO: meta-oe ... PASS
> > > > INFO: meta-networking ... PASS
> > >
> > > We merged this but have started seeing:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/166
> > >
> > > which seems a little strange and is probably related to this series.
> > >
> >
> >
> > yes, it's related.
> >
> > ...
> > INFO: Adding meta-poky to the list of layers to test, as a dependency
> > ...
> > INFO: meta-poky ... SKIPPED (Layer under test should not present in
> BBLAYERS)
> >
> > With these changes, we are now adding all dependencies and trying to
> test them too. But meta-poky seems to
> > be in bblayers.conf, and yocto-check-layer won't work since it needs to
> compute all checksum *without* the
> > layer in the config first.
> >
> > It is probably a good idea to make sure bblayers only has oe-core when
> we try to perform YP compat.
> > Alternatively, as discussed, we use --no-auto-dependency on AB..
>
> Fair enough, I'll update the config to do that.
>

These patches are now merged in master, and I believe they should be merged
in dunfell and hardknott too. The 4 patches apply as-is in both branches.
Let me know if you need anything from me!



>
> Cheers,
>
> Richard
>
> >
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 4101 bytes --]

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Steve Sakoman]

On Mon, Aug 2, 2021 at 12:35 AM Nicolas Dechesne
<nicolas.dechesne@linaro.org> wrote:
>
> Richard, Steve, Anuj,
>
>
> On Fri, Jul 30, 2021 at 11:28 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>>
>> On Fri, 2021-07-30 at 11:08 +0200, Nicolas Dechesne wrote:
>> >
>> >
>> > On Fri, Jul 30, 2021 at 10:03 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>> > > On Thu, 2021-07-22 at 14:46 +0200, Nicolas Dechesne wrote:
>> > > > It has been recentely mentioned to me by the YP TSC that a layer must
>> > > > not only pass the yocto-check-layer script, but also all its
>> > > > dependencies. These patches add support for testing dependencies
>> > > > automatically.
>> > > >
>> > > > Patch #1 and #3 are not much related, but improvements I made along
>> > > > the lines.
>> > > >
>> > > > Patch #2 split the internal function into 2 parts, so that we can get
>> > > > the list of dependencies from the main script.
>> > > >
>> > > > Patch #4 is where the processing of the dependencies is done. I chose
>> > > > to enable checking the dependencies by default, and use
>> > > > '--no-auto-dependency' to disable it, we could decide the other way
>> > > > around. For any dependency of the 'layers under test' we will simply
>> > > > add them to the list of layers to test unless they are already there.
>> > > >
>> > > > Here is a sample output with these changes:
>> > > >
>> > > > $ yocto-check-layer /work/oe/sources/meta-aws \
>> > > >                     --dependency /work/oe/sources/meta-openembedded/
>> > > >
>> > > > INFO: Summary of results:
>> > > > INFO:
>> > > > INFO: meta-aws ... PASS
>> > > > INFO: meta-python ... PASS
>> > > > INFO: meta-oe ... PASS
>> > > > INFO: meta-networking ... PASS
>> > >
>> > > We merged this but have started seeing:
>> > >
>> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/166
>> > >
>> > > which seems a little strange and is probably related to this series.
>> > >
>> >
>> >
>> > yes, it's related.
>> >
>> > ...
>> > INFO: Adding meta-poky to the list of layers to test, as a dependency
>> > ...
>> > INFO: meta-poky ... SKIPPED (Layer under test should not present in BBLAYERS)
>> >
>> > With these changes, we are now adding all dependencies and trying to test them too. But meta-poky seems to
>> > be in bblayers.conf, and yocto-check-layer won't work since it needs to compute all checksum *without* the
>> > layer in the config first.
>> >
>> > It is probably a good idea to make sure bblayers only has oe-core when we try to perform YP compat.
>> > Alternatively, as discussed, we use --no-auto-dependency on AB..
>>
>> Fair enough, I'll update the config to do that.
>
>
> These patches are now merged in master, and I believe they should be merged in dunfell and hardknott too. The 4 patches apply as-is in both branches. Let me know if you need anything from me!

Done.  I assume I also need a version of the autobuilder-helper patch:

https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/commit/?id=fe72e222961d69dcd4d638e7e05b437a65e8f34c

It seems that dunfell is checking a much more limited set of layers
(only meta-intel for check-layer-nightly)  Should I expand this list?

Steve


>
>>
>>
>> Cheers,
>>
>> Richard
>>
>> >
>>
>>
>>

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Richard Purdie]

On Mon, 2021-08-02 at 04:57 -1000, Steve Sakoman wrote:
> On Mon, Aug 2, 2021 at 12:35 AM Nicolas Dechesne
> <nicolas.dechesne@linaro.org> wrote:
> > 
> > These patches are now merged in master, and I believe they should be merged in 
> > dunfell and hardknott too. The 4 patches apply as-is in both branches. Let me 
> > know if you need anything from me!
> 
> Done.  I assume I also need a version of the autobuilder-helper patch:
> 
> https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-
> helper/commit/?id=fe72e222961d69dcd4d638e7e05b437a65e8f34c
> 
> It seems that dunfell is checking a much more limited set of layers
> (only meta-intel for check-layer-nightly)  Should I expand this list?

There has been work to get some of them working, so yes, I think it would
be good to test and see where we stand.

Cheers,

Richard

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Steve Sakoman]

On Mon, Aug 2, 2021 at 5:04 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2021-08-02 at 04:57 -1000, Steve Sakoman wrote:
> > On Mon, Aug 2, 2021 at 12:35 AM Nicolas Dechesne
> > <nicolas.dechesne@linaro.org> wrote:
> > >
> > > These patches are now merged in master, and I believe they should be merged in
> > > dunfell and hardknott too. The 4 patches apply as-is in both branches. Let me
> > > know if you need anything from me!
> >
> > Done.  I assume I also need a version of the autobuilder-helper patch:
> >
> > https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-
> > helper/commit/?id=fe72e222961d69dcd4d638e7e05b437a65e8f34c
> >
> > It seems that dunfell is checking a much more limited set of layers
> > (only meta-intel for check-layer-nightly)  Should I expand this list?
>
> There has been work to get some of them working, so yes, I think it would
> be good to test and see where we stand.

OK, will do.

Steve

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Denys Dmytriyenko]

On Mon, Aug 02, 2021 at 05:07:41AM -1000, Steve Sakoman wrote:
> On Mon, Aug 2, 2021 at 5:04 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > On Mon, 2021-08-02 at 04:57 -1000, Steve Sakoman wrote:
> > > On Mon, Aug 2, 2021 at 12:35 AM Nicolas Dechesne
> > > <nicolas.dechesne@linaro.org> wrote:
> > > >
> > > > These patches are now merged in master, and I believe they should be merged in
> > > > dunfell and hardknott too. The 4 patches apply as-is in both branches. Let me
> > > > know if you need anything from me!
> > >
> > > Done.  I assume I also need a version of the autobuilder-helper patch:
> > >
> > > https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-
> > > helper/commit/?id=fe72e222961d69dcd4d638e7e05b437a65e8f34c
> > >
> > > It seems that dunfell is checking a much more limited set of layers
> > > (only meta-intel for check-layer-nightly)  Should I expand this list?
> >
> > There has been work to get some of them working, so yes, I think it would
> > be good to test and see where we stand.
> 
> OK, will do.

Steve,

Would you be able to publish results of your testing on dunfell?

I'm interested in some expanded compat checks in master, but they were blocked 
by known limitations in dunfell - was wondering if those got resolved yet. 
Thanks!

-- 
Regards,
Denys Dmytriyenko <denis@denix.org>
PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186  6D76 4209 0272 9A92 C964

Re: [OE-core] [PATCH 0/4] yocto-check-layer: add support to check for dependencies

[Steve Sakoman]

On Tue, Aug 3, 2021 at 4:14 PM Denys Dmytriyenko <denis@denix.org> wrote:
>
> On Mon, Aug 02, 2021 at 05:07:41AM -1000, Steve Sakoman wrote:
> > On Mon, Aug 2, 2021 at 5:04 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > >
> > > On Mon, 2021-08-02 at 04:57 -1000, Steve Sakoman wrote:
> > > > On Mon, Aug 2, 2021 at 12:35 AM Nicolas Dechesne
> > > > <nicolas.dechesne@linaro.org> wrote:
> > > > >
> > > > > These patches are now merged in master, and I believe they should be merged in
> > > > > dunfell and hardknott too. The 4 patches apply as-is in both branches. Let me
> > > > > know if you need anything from me!
> > > >
> > > > Done.  I assume I also need a version of the autobuilder-helper patch:
> > > >
> > > > https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-
> > > > helper/commit/?id=fe72e222961d69dcd4d638e7e05b437a65e8f34c
> > > >
> > > > It seems that dunfell is checking a much more limited set of layers
> > > > (only meta-intel for check-layer-nightly)  Should I expand this list?
> > >
> > > There has been work to get some of them working, so yes, I think it would
> > > be good to test and see where we stand.
> >
> > OK, will do.
>
> Steve,
>
> Would you be able to publish results of your testing on dunfell?
>
> I'm interested in some expanded compat checks in master, but they were blocked
> by known limitations in dunfell - was wondering if those got resolved yet.

I was able to successfully add meta-aws, meta-openembedded and
meta-virtualization to dunfell check-layer-nightly:

https://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/commit/?h=contrib/sakoman&id=f0108c3cdb8d50dfb877ff880fbc1a45e9abe899

The other layers currently in master (meta-agl, meta-arm and meta-ti)
failed with dunfell.

I'd be happy to add those to dunfell check-layer-nightly once they are fixed.

Steve