* [PATCH] create-spdx: Don't collect natives sources
@ 2021-09-22 22:14 Saul Wold
2021-09-23 8:23 ` [OE-core] " Konrad Weihmann
0 siblings, 1 reply; 5+ messages in thread
From: Saul Wold @ 2021-09-22 22:14 UTC (permalink / raw)
To: openembedded-core, JPEWhacker
When the collect_dep_sources() runs, it collects sources from both native
and non-native recipes. Later when the GENERATED_FROM matching occurs it
may find the file (via checksum) from the native recipe since it's the
same checksum as the target file. The that are generated DocumentRefs
point to the native recipe rather than the target recipe DocumentRef.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
---
meta/classes/create-spdx.bbclass | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
index 3c73c21c04..8f7fae43f1 100644
--- a/meta/classes/create-spdx.bbclass
+++ b/meta/classes/create-spdx.bbclass
@@ -336,6 +336,10 @@ def collect_dep_sources(d, dep_recipes):
sources = {}
for dep in dep_recipes:
+ # Don't collect sources from native recipes as they
+ # match non-native sources also.
+ if "-native" in dep.recipe.name:
+ continue
recipe_files = set(dep.recipe.hasFiles)
for spdx_file in dep.doc.files:
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [OE-core] [PATCH] create-spdx: Don't collect natives sources
2021-09-22 22:14 [PATCH] create-spdx: Don't collect natives sources Saul Wold
@ 2021-09-23 8:23 ` Konrad Weihmann
2021-09-23 10:27 ` Peter Kjellerstedt
0 siblings, 1 reply; 5+ messages in thread
From: Konrad Weihmann @ 2021-09-23 8:23 UTC (permalink / raw)
To: Saul Wold, openembedded-core, JPEWhacker
On 23.09.21 00:14, Saul Wold wrote:
> When the collect_dep_sources() runs, it collects sources from both native
> and non-native recipes. Later when the GENERATED_FROM matching occurs it
> may find the file (via checksum) from the native recipe since it's the
> same checksum as the target file. The that are generated DocumentRefs
> point to the native recipe rather than the target recipe DocumentRef.
>
> Signed-off-by: Saul Wold <saul.wold@windriver.com>
> ---
> meta/classes/create-spdx.bbclass | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
> index 3c73c21c04..8f7fae43f1 100644
> --- a/meta/classes/create-spdx.bbclass
> +++ b/meta/classes/create-spdx.bbclass
> @@ -336,6 +336,10 @@ def collect_dep_sources(d, dep_recipes):
>
> sources = {}
> for dep in dep_recipes:
> + # Don't collect sources from native recipes as they
> + # match non-native sources also.
> + if "-native" in dep.recipe.name:
That looks a bit too fuzzy to me - I would have expected
if dep.recipe.name.endswith("-native"):
here. Otherwise this is very prone to false positives
> + continue
> recipe_files = set(dep.recipe.hasFiles)
>
> for spdx_file in dep.doc.files:
>
>
>
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [OE-core] [PATCH] create-spdx: Don't collect natives sources
2021-09-23 8:23 ` [OE-core] " Konrad Weihmann
@ 2021-09-23 10:27 ` Peter Kjellerstedt
2021-09-23 12:16 ` Joshua Watt
0 siblings, 1 reply; 5+ messages in thread
From: Peter Kjellerstedt @ 2021-09-23 10:27 UTC (permalink / raw)
To: Konrad Weihmann, Saul Wold, openembedded-core, JPEWhacker
> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Konrad Weihmann
> Sent: den 23 september 2021 10:24
> To: Saul Wold <Saul.Wold@windriver.com>; openembedded-
> core@lists.openembedded.org; JPEWhacker@gmail.com
> Subject: Re: [OE-core] [PATCH] create-spdx: Don't collect natives sources
>
>
>
> On 23.09.21 00:14, Saul Wold wrote:
> > When the collect_dep_sources() runs, it collects sources from both
> native
> > and non-native recipes. Later when the GENERATED_FROM matching occurs it
> > may find the file (via checksum) from the native recipe since it's the
> > same checksum as the target file. The that are generated DocumentRefs
> > point to the native recipe rather than the target recipe DocumentRef.
> >
> > Signed-off-by: Saul Wold <saul.wold@windriver.com>
> > ---
> > meta/classes/create-spdx.bbclass | 4 ++++
> > 1 file changed, 4 insertions(+)
> >
> > diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-
> spdx.bbclass
> > index 3c73c21c04..8f7fae43f1 100644
> > --- a/meta/classes/create-spdx.bbclass
> > +++ b/meta/classes/create-spdx.bbclass
> > @@ -336,6 +336,10 @@ def collect_dep_sources(d, dep_recipes):
> >
> > sources = {}
> > for dep in dep_recipes:
> > + # Don't collect sources from native recipes as they
> > + # match non-native sources also.
> > + if "-native" in dep.recipe.name:
>
> That looks a bit too fuzzy to me - I would have expected
>
> if dep.recipe.name.endswith("-native"):
>
> here. Otherwise this is very prone to false positives
Absolutely. E.g., we have a recipe called crate-native-tls for
the native-tls Rust crate.
At the same time I hate all these hardcoded tests for recipe types
all over OE Core as they assume only the types that are in OE Core
exist without any (easy) way of extending the tests for non-OE Core
recipe types.
Would it instead be possible to add the is_native in do_create_spdx()
to the recipe object so that it is available here?
> > + continue
> > recipe_files = set(dep.recipe.hasFiles)
> >
> > for spdx_file in dep.doc.files:
//Peter
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [OE-core] [PATCH] create-spdx: Don't collect natives sources
2021-09-23 10:27 ` Peter Kjellerstedt
@ 2021-09-23 12:16 ` Joshua Watt
2021-09-23 14:02 ` Saul Wold
0 siblings, 1 reply; 5+ messages in thread
From: Joshua Watt @ 2021-09-23 12:16 UTC (permalink / raw)
To: Peter Kjellerstedt; +Cc: Konrad Weihmann, Saul Wold, OE-core
[-- Attachment #1: Type: text/plain, Size: 2566 bytes --]
On Thu, Sep 23, 2021, 5:27 AM Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-
> > core@lists.openembedded.org> On Behalf Of Konrad Weihmann
> > Sent: den 23 september 2021 10:24
> > To: Saul Wold <Saul.Wold@windriver.com>; openembedded-
> > core@lists.openembedded.org; JPEWhacker@gmail.com
> > Subject: Re: [OE-core] [PATCH] create-spdx: Don't collect natives sources
> >
> >
> >
> > On 23.09.21 00:14, Saul Wold wrote:
> > > When the collect_dep_sources() runs, it collects sources from both
> > native
> > > and non-native recipes. Later when the GENERATED_FROM matching occurs
> it
> > > may find the file (via checksum) from the native recipe since it's the
> > > same checksum as the target file. The that are generated DocumentRefs
> > > point to the native recipe rather than the target recipe DocumentRef.
> > >
> > > Signed-off-by: Saul Wold <saul.wold@windriver.com>
> > > ---
> > > meta/classes/create-spdx.bbclass | 4 ++++
> > > 1 file changed, 4 insertions(+)
> > >
> > > diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-
> > spdx.bbclass
> > > index 3c73c21c04..8f7fae43f1 100644
> > > --- a/meta/classes/create-spdx.bbclass
> > > +++ b/meta/classes/create-spdx.bbclass
> > > @@ -336,6 +336,10 @@ def collect_dep_sources(d, dep_recipes):
> > >
> > > sources = {}
> > > for dep in dep_recipes:
> > > + # Don't collect sources from native recipes as they
> > > + # match non-native sources also.
> > > + if "-native" in dep.recipe.name:
> >
> > That looks a bit too fuzzy to me - I would have expected
> >
> > if dep.recipe.name.endswith("-native"):
> >
> > here. Otherwise this is very prone to false positives
>
> Absolutely. E.g., we have a recipe called crate-native-tls for
> the native-tls Rust crate.
>
> At the same time I hate all these hardcoded tests for recipe types
> all over OE Core as they assume only the types that are in OE Core
> exist without any (easy) way of extending the tests for non-OE Core
> recipe types.
>
> Would it instead be possible to add the is_native in do_create_spdx()
> to the recipe object so that it is available here?
>
Yes. We should be able to add a custom annotation when we create the recipe
that we can check later
> > > + continue
> > > recipe_files = set(dep.recipe.hasFiles)
> > >
> > > for spdx_file in dep.doc.files:
>
> //Peter
>
>
[-- Attachment #2: Type: text/html, Size: 4088 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [OE-core] [PATCH] create-spdx: Don't collect natives sources
2021-09-23 12:16 ` Joshua Watt
@ 2021-09-23 14:02 ` Saul Wold
0 siblings, 0 replies; 5+ messages in thread
From: Saul Wold @ 2021-09-23 14:02 UTC (permalink / raw)
To: Joshua Watt, Peter Kjellerstedt; +Cc: Konrad Weihmann, OE-core
On 9/23/21 5:16 AM, Joshua Watt wrote:
>
>
> On Thu, Sep 23, 2021, 5:27 AM Peter Kjellerstedt
> <peter.kjellerstedt@axis.com <mailto:peter.kjellerstedt@axis.com>> wrote:
>
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org
> <mailto:openembedded-core@lists.openembedded.org> <openembedded-
> > core@lists.openembedded.org <mailto:core@lists.openembedded.org>>
> On Behalf Of Konrad Weihmann
> > Sent: den 23 september 2021 10:24
> > To: Saul Wold <Saul.Wold@windriver.com
> <mailto:Saul.Wold@windriver.com>>; openembedded-
> > core@lists.openembedded.org <mailto:core@lists.openembedded.org>;
> JPEWhacker@gmail.com <mailto:JPEWhacker@gmail.com>
> > Subject: Re: [OE-core] [PATCH] create-spdx: Don't collect natives
> sources
> >
> >
> >
> > On 23.09.21 00:14, Saul Wold wrote:
> > > When the collect_dep_sources() runs, it collects sources from both
> > native
> > > and non-native recipes. Later when the GENERATED_FROM matching
> occurs it
> > > may find the file (via checksum) from the native recipe since
> it's the
> > > same checksum as the target file. The that are generated
> DocumentRefs
> > > point to the native recipe rather than the target recipe
> DocumentRef.
> > >
> > > Signed-off-by: Saul Wold <saul.wold@windriver.com
> <mailto:saul.wold@windriver.com>>
> > > ---
> > > meta/classes/create-spdx.bbclass | 4 ++++
> > > 1 file changed, 4 insertions(+)
> > >
> > > diff --git a/meta/classes/create-spdx.bbclass
> b/meta/classes/create-
> > spdx.bbclass
> > > index 3c73c21c04..8f7fae43f1 100644
> > > --- a/meta/classes/create-spdx.bbclass
> > > +++ b/meta/classes/create-spdx.bbclass
> > > @@ -336,6 +336,10 @@ def collect_dep_sources(d, dep_recipes):
> > >
> > > sources = {}
> > > for dep in dep_recipes:
> > > + # Don't collect sources from native recipes as they
> > > + # match non-native sources also.
> > > + if "-native" in dep.recipe.name
> <https://urldefense.com/v3/__http://dep.recipe.name__;!!AjveYdw8EvQ!KhzFBhUyb3qZAcSIhlUVGRaWHWWBWh2YZu-s_7VeaS3HaafxJOuneeo9xWwBCOboCok$>:
> >
> > That looks a bit too fuzzy to me - I would have expected
> >
> > if dep.recipe.name.endswith("-native"):
> >
> > here. Otherwise this is very prone to false positives
>
> Absolutely. E.g., we have a recipe called crate-native-tls for
> the native-tls Rust crate.
>
> At the same time I hate all these hardcoded tests for recipe types
> all over OE Core as they assume only the types that are in OE Core
> exist without any (easy) way of extending the tests for non-OE Core
> recipe types.
>
> Would it instead be possible to add the is_native in do_create_spdx()
> to the recipe object so that it is available here?
>
>
> Yes. We should be able to add a custom annotation when we create the
> recipe that we can check later
>
I will re-work the patch and work on adding the is_native to the Recipe
object.
Sau!
>
> > > + continue
> > > recipe_files = set(dep.recipe.hasFiles)
> > >
> > > for spdx_file in dep.doc.files:
>
> //Peter
>
--
Sau!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-09-23 14:02 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-22 22:14 [PATCH] create-spdx: Don't collect natives sources Saul Wold
2021-09-23 8:23 ` [OE-core] " Konrad Weihmann
2021-09-23 10:27 ` Peter Kjellerstedt
2021-09-23 12:16 ` Joshua Watt
2021-09-23 14:02 ` Saul Wold
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.