[PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Tomasz Dziendzielski]

If image recipe depends on another image recipe it will try to use
license recipeinfo that is not deployed. It will result in:
| Exception: FileNotFoundError: [Errno 2] No such file or directory:
| 'TMPDIR/deploy/licenses/foo-image/recipeinfo'

We can't determine in a simple and efficient way if dependency recipe
inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
variable that will contain list of recipes that should be skipped.

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
---
 meta/classes/license_image.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/license_image.bbclass b/meta/classes/license_image.bbclass
index 8fd88cfb2d..e2f3178306 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
     pn = d.getVar("PN", True)
     depends = list(set([dep[0] for dep
                     in list(taskdata.values())
-                    if not dep[0].endswith("-native") and not dep[0] == pn]))
+                    if not dep[0].endswith("-native") and not dep[0] == pn
+                    and dep[0] not in (d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))
 
     # To verify what was deployed it checks the rootfs dependencies against
     # the SSTATE_MANIFESTS for "deploy" task.
-- 
2.30.0

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Michael Ho]

Hi,

I submitted something similar awhile back to this mailing list.

https://lists.openembedded.org/g/openembedded-core/message/146607?p=,,,20,0,0,0::relevance,,Michael.Ho%40bmw.de,20,2,0,79619287

If yours makes more sense (I didn't think there's ever a case where an image
recipe actually has a license) then my patch should probably be skipped over
for yours.

Kind regards,
Michael Ho

-- 
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm

Tel.: ­+49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael.ho@bmw-carit.de 
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-------------------------------------------------------------------------
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-------------------------------------------------------------------------
 

On 04.02.21, 13:11, "openembedded-core@lists.openembedded.org on behalf of Tomasz Dziendzielski" <openembedded-core@lists.openembedded.org on behalf of tomasz.dziendzielski@gmail.com> wrote:

    If image recipe depends on another image recipe it will try to use
    license recipeinfo that is not deployed. It will result in:
    | Exception: FileNotFoundError: [Errno 2] No such file or directory:
    | 'TMPDIR/deploy/licenses/foo-image/recipeinfo'

    We can't determine in a simple and efficient way if dependency recipe
    inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
    variable that will contain list of recipes that should be skipped.

    Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
    ---
     meta/classes/license_image.bbclass | 3 ++-
     1 file changed, 2 insertions(+), 1 deletion(-)

    diff --git a/meta/classes/license_image.bbclass b/meta/classes/license_image.bbclass
    index 8fd88cfb2d..e2f3178306 100644
    --- a/meta/classes/license_image.bbclass
    +++ b/meta/classes/license_image.bbclass
    @@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
         pn = d.getVar("PN", True)
         depends = list(set([dep[0] for dep
                         in list(taskdata.values())
    -                    if not dep[0].endswith("-native") and not dep[0] == pn]))
    +                    if not dep[0].endswith("-native") and not dep[0] == pn
    +                    and dep[0] not in (d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))

         # To verify what was deployed it checks the rootfs dependencies against
         # the SSTATE_MANIFESTS for "deploy" task.
    -- 
    2.30.0

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Michael Ho]

(Sorry, didn't read your patch correctly, it already included my change).

I guess I mean should my change (not dep[0] == pn) be removed and yours
take its place as yours should cover that case.

Kind regards,
Michael Ho

-- 
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm

Tel.: ­+49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael.ho@bmw-carit.de 
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-------------------------------------------------------------------------
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-------------------------------------------------------------------------
 

On 04.02.21, 14:45, "openembedded-core@lists.openembedded.org on behalf of bounce+67612+147659+4454854+8786752@lists.openembedded.org" <openembedded-core@lists.openembedded.org on behalf of bounce+67612+147659+4454854+8786752@lists.openembedded.org> wrote:

    Hi,

    I submitted something similar awhile back to this mailing list.

    https://lists.openembedded.org/g/openembedded-core/message/146607?p=,,,20,0,0,0::relevance,,Michael.Ho%40bmw.de,20,2,0,79619287

    If yours makes more sense (I didn't think there's ever a case where an image
    recipe actually has a license) then my patch should probably be skipped over
    for yours.

    Kind regards,
    Michael Ho

    -- 
    BMW Car IT GmbH
    Michael Ho
    Spezialist Entwicklung – Build and Release Engineering
    Lise-Meitner-Straße 14
    89081 Ulm

    Tel.: ­+49-731-37804-071
    Mobil: +49-152-54980-471
    Fax: +49-731-37804-001
    Mail: michael.ho@bmw-carit.de 
    Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
    -------------------------------------------------------------------------
    BMW Car IT GmbH
    Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
    Sitz und Registergericht: München HRB 134810
    -------------------------------------------------------------------------


    On 04.02.21, 13:11, "openembedded-core@lists.openembedded.org on behalf of Tomasz Dziendzielski" <openembedded-core@lists.openembedded.org on behalf of tomasz.dziendzielski@gmail.com> wrote:

        If image recipe depends on another image recipe it will try to use
        license recipeinfo that is not deployed. It will result in:
        | Exception: FileNotFoundError: [Errno 2] No such file or directory:
        | 'TMPDIR/deploy/licenses/foo-image/recipeinfo'

        We can't determine in a simple and efficient way if dependency recipe
        inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
        variable that will contain list of recipes that should be skipped.

        Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
        ---
         meta/classes/license_image.bbclass | 3 ++-
         1 file changed, 2 insertions(+), 1 deletion(-)

        diff --git a/meta/classes/license_image.bbclass b/meta/classes/license_image.bbclass
        index 8fd88cfb2d..e2f3178306 100644
        --- a/meta/classes/license_image.bbclass
        +++ b/meta/classes/license_image.bbclass
        @@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
             pn = d.getVar("PN", True)
             depends = list(set([dep[0] for dep
                             in list(taskdata.values())
        -                    if not dep[0].endswith("-native") and not dep[0] == pn]))
        +                    if not dep[0].endswith("-native") and not dep[0] == pn
        +                    and dep[0] not in (d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))

             # To verify what was deployed it checks the rootfs dependencies against
             # the SSTATE_MANIFESTS for "deploy" task.
        -- 
        2.30.0

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Tomasz Dziendzielski]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

 >I guess I mean should my change (not dep[0] == pn) be removed and yours
>take its place as yours should cover that case.

Hi Michael,
I would say we should keep both. Your change would be enough for most cases
and most developers wouldn't even need to care about that problem.  I think
image recipes depending on other image recipes are less common scenarios
and only in these cases my change would be needed. If we remove "(not
dep[0] == pn)" we would require all developers to adapt their builds with
SKIP_LICENSE_MANIFEST_RECIPES variable.

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 706 bytes --]

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Bruce Ashfield]

On Thu, Feb 4, 2021 at 7:10 AM Tomasz Dziendzielski
<tomasz.dziendzielski@gmail.com> wrote:
>
> If image recipe depends on another image recipe it will try to use
> license recipeinfo that is not deployed. It will result in:
> | Exception: FileNotFoundError: [Errno 2] No such file or directory:
> | 'TMPDIR/deploy/licenses/foo-image/recipeinfo'
>
> We can't determine in a simple and efficient way if dependency recipe
> inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
> variable that will contain list of recipes that should be skipped.
>
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> ---
>  meta/classes/license_image.bbclass | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/license_image.bbclass b/meta/classes/license_image.bbclass
> index 8fd88cfb2d..e2f3178306 100644
> --- a/meta/classes/license_image.bbclass
> +++ b/meta/classes/license_image.bbclass
> @@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
>      pn = d.getVar("PN", True)
>      depends = list(set([dep[0] for dep
>                      in list(taskdata.values())
> -                    if not dep[0].endswith("-native") and not dep[0] == pn]))
> +                    if not dep[0].endswith("-native") and not dep[0] == pn
> +                    and dep[0] not in (d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))

I didn't see anything on the docs mailing list, but we should also
have a matching update to the docs for the new variable (unless it is
already in the docs, I didn't check for that).

Cheers,

Bruce


>
>      # To verify what was deployed it checks the rootfs dependencies against
>      # the SSTATE_MANIFESTS for "deploy" task.
> --
> 2.30.0
>
>
> 
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Tomasz Dziendzielski]

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]

>I didn't see anything on the docs mailing list, but we should also
>have a matching update to the docs for the new variable (unless it is
>already in the docs, I didn't check for that).

Thanks, please check now:
https://lists.yoctoproject.org/g/docs/message/860

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 496 bytes --]

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Richard Purdie]

On Thu, 2021-02-04 at 13:13 +0100, Tomasz Dziendzielski wrote:
> If image recipe depends on another image recipe it will try to use
> license recipeinfo that is not deployed. It will result in:
> > Exception: FileNotFoundError: [Errno 2] No such file or directory:
> > 'TMPDIR/deploy/licenses/foo-image/recipeinfo'
> 
> We can't determine in a simple and efficient way if dependency recipe
> inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
> variable that will contain list of recipes that should be skipped.
> 
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> ---
>  meta/classes/license_image.bbclass | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/license_image.bbclass b/meta/classes/license_image.bbclass
> index 8fd88cfb2d..e2f3178306 100644
> --- a/meta/classes/license_image.bbclass
> +++ b/meta/classes/license_image.bbclass
> @@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
>      pn = d.getVar("PN", True)
>      depends = list(set([dep[0] for dep
>                      in list(taskdata.values())
> -                    if not dep[0].endswith("-native") and not dep[0] == pn]))
> +                    if not dep[0].endswith("-native") and not dep[0] == pn
> +                    and dep[0] not in (d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))
>  
> 
> 
> 

I've been giving this some thought and I really don't like having to
remember to manually add special cases like this, it sounds/looks like
something we should really fix in a better way.

Perhaps we should just skip license files if they don't exist? We could
perhaps show a warning if the match isn't for a recipe called *-image?

Cheers,

Richard

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Tomasz Dziendzielski]

[-- Attachment #1: Type: text/plain, Size: 1093 bytes --]

Thanks for the answer.

>I've been giving this some thought and I really don't like having to
>remember to manually add special cases like this, it sounds/looks like
>something we should really fix in a better way.

The other way that comes to my mind would be to loop over all dependencies
and check if they inherit image.bbclass which would increase build time.
Another problem is that we don't have direct access to other recipes'
metadata, so we would need to prepare such mechanism.

>Perhaps we should just skip license files if they don't exist? We could
>perhaps show a warning if the match isn't for a recipe called *-image?

There might be recipes inheriting image.bbclass that don't have "image" in
its name (which is also my case). I can prepare the change to just print a
warning instead of error if the recipe can't find a license manifest for
any recipe, not only the ones called *-image. Unfortunately that way we
could not catch missing manifests with recipes that should provide them.
Would such change be accepted?

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 1335 bytes --]

Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes

[Richard Purdie]

On Tue, 2021-02-09 at 16:37 +0100, Tomasz Dziendzielski wrote:
> 
> Thanks for the answer.
> 
> > I've been giving this some thought and I really don't like having
> > to
> > remember to manually add special cases like this, it sounds/looks
> > like
> > something we should really fix in a better way.
> 
> The other way that comes to my mind would be to loop over all
> dependencies and check if they inherit image.bbclass which would
> increase build time. Another problem is that we don't have direct
> access to other recipes' metadata, so we would need to prepare such
> mechanism.

That isn't something we want to support.

> > Perhaps we should just skip license files if they don't exist? We
> > could
> > perhaps show a warning if the match isn't for a recipe called *-
> > image?
> 
> There might be recipes inheriting image.bbclass that don't have
> "image" in its name (which is also my case). I can prepare the change
> to just print a warning instead of error if the recipe can't find a
> license manifest for any recipe, not only the ones called *-image.
> Unfortunately that way we could not catch missing manifests with
> recipes that should provide them. Would such change be accepted?

I think the warn/error issue isn't the real solution, "good" builds
shouldn't show warnings and this case would.

I guess the other option is to enable populate_lic for image recipes
rather than deleting the task, then there would be some kind of license
information available. Not sure whether that makes sense or not (and
whether it should be the image recipe license or the sum of the
licences making up that image is a secondary difficult question)...

Cheers,

Richard