[PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Jaewon Lee]

When using a recipe space kernel-meta, scc files are added through
SRC_URI, but they may include corresponding kernel fragments that are
not necessarily in SRC_URI.

For bitbake, this is not a problem because the kernel-yocto class adds
the path where the .scc file was found to includes which consequentially
makes the .cfg file available to the kernel build.

However, when using devtool, only files specified in SRC_URI are copied
to oe-local-files in devtool's workspace. So if the cfg file is not in
SRC_URI, it won't be copied, causing a kernel build failure when trying
to find it.

This fix parses local .scc files in SRC_URI, copies the corresponding .cfg
file to devtool's workdir, and also adds it to local_files so it is
available when doing a devtool build for the kernel.

[YOCTO #12858]

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
---
 meta/classes/devtool-source.bbclass | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
index 56882a4..c70fea2 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -90,11 +90,23 @@ python devtool_post_unpack() {
                         fname in files])
         return ret
 
+    is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d)
     # Move local source files into separate subdir
     recipe_patches = [os.path.basename(patch) for patch in
                         oe.recipeutils.get_recipe_patches(d)]
     local_files = oe.recipeutils.get_recipe_local_files(d)
 
+    if is_kernel_yocto:
+      for key in local_files.copy():
+        if key.endswith('scc'):
+          sccfile = open(local_files[key], 'r')
+          for l in sccfile:
+            line = l.split()
+            if line and line[0] == 'kconf' and line[-1].endswith('.cfg'):
+              local_files[line[-1]] = os.path.join(os.path.dirname(local_files[key]), line[-1])
+              shutil.copy2(os.path.join(os.path.dirname(local_files[key]), line[-1]), workdir)
+          sccfile.close()
+
     # Ignore local files with subdir={BP}
     srcabspath = os.path.abspath(srcsubdir)
     local_files = [fname for fname in local_files if
-- 
2.7.4

[PATCH] kernel-yocto.bbclass: Adds oe-local-files path (devtool) to include directives

[Jaewon Lee]

The devtool-source class moves all local files specified in SRC_URI to
an oe-local-files directory. When using devtool and a recipe space kernel-meta,
devtool modify throws an error because the paths the kernel-yocto class
is looking for feature directories in, don't include the oe-local-files
directory which devtool is using.

This patch checks for feature directories in oe-local-files,
and if present, adds that path to include directives.

[YOCTO #12855]

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
---
 meta/classes/kernel-yocto.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 82d8074..077a1ab 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -140,6 +140,8 @@ do_kernel_metadata() {
 			includes="$includes -I${WORKDIR}/$f/kernel-meta"
 	        elif [ -d "${WORKDIR}/$f" ]; then
 			includes="$includes -I${WORKDIR}/$f"
+		elif [ -d "${WORKDIR}/../oe-local-files/$f" ]; then
+			includes="$includes -I${WORKDIR}/../oe-local-files/$f"
 		fi
 	done
 	for s in ${sccs} ${patches}; do
-- 
2.7.4

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Anuj Mittal]

On 07/31/2018 05:21 AM, Jaewon Lee wrote:
> When using a recipe space kernel-meta, scc files are added through
> SRC_URI, but they may include corresponding kernel fragments that are
> not necessarily in SRC_URI.
> 
> For bitbake, this is not a problem because the kernel-yocto class adds
> the path where the .scc file was found to includes which consequentially
> makes the .cfg file available to the kernel build.
> 
> However, when using devtool, only files specified in SRC_URI are copied
> to oe-local-files in devtool's workspace. So if the cfg file is not in
> SRC_URI, it won't be copied, causing a kernel build failure when trying
> to find it.
> 
> This fix parses local .scc files in SRC_URI, copies the corresponding .cfg
> file to devtool's workdir, and also adds it to local_files so it is
> available when doing a devtool build for the kernel.
> 
> [YOCTO #12858]
> 
> Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
> ---
>  meta/classes/devtool-source.bbclass | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
> index 56882a4..c70fea2 100644
> --- a/meta/classes/devtool-source.bbclass
> +++ b/meta/classes/devtool-source.bbclass
> @@ -90,11 +90,23 @@ python devtool_post_unpack() {
>                          fname in files])
>          return ret
>  
> +    is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d)
>      # Move local source files into separate subdir
>      recipe_patches = [os.path.basename(patch) for patch in
>                          oe.recipeutils.get_recipe_patches(d)]
>      local_files = oe.recipeutils.get_recipe_local_files(d)
>  
> +    if is_kernel_yocto:
> +      for key in local_files.copy():
> +        if key.endswith('scc'):
> +          sccfile = open(local_files[key], 'r')
> +          for l in sccfile:
> +            line = l.split()
> +            if line and line[0] == 'kconf' and line[-1].endswith('.cfg'):
> +              local_files[line[-1]] = os.path.join(os.path.dirname(local_files[key]), line[-1])
> +              shutil.copy2(os.path.join(os.path.dirname(local_files[key]), line[-1]), workdir)
> +          sccfile.close()
> +

Would the patches included in these .scc files also need to be handled
in the same way? Would this also work if there are other scc files
included in a scc file?

Thanks,

Anuj

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Alejandro Enedino Hernandez Samaniego]

Hey Anuj,


On 08/01/2018 04:25 AM, Anuj Mittal wrote:
> On 07/31/2018 05:21 AM, Jaewon Lee wrote:
>> When using a recipe space kernel-meta, scc files are added through
>> SRC_URI, but they may include corresponding kernel fragments that are
>> not necessarily in SRC_URI.
>>
>> For bitbake, this is not a problem because the kernel-yocto class adds
>> the path where the .scc file was found to includes which consequentially
>> makes the .cfg file available to the kernel build.
>>
>> However, when using devtool, only files specified in SRC_URI are copied
>> to oe-local-files in devtool's workspace. So if the cfg file is not in
>> SRC_URI, it won't be copied, causing a kernel build failure when trying
>> to find it.
>>
>> This fix parses local .scc files in SRC_URI, copies the corresponding .cfg
>> file to devtool's workdir, and also adds it to local_files so it is
>> available when doing a devtool build for the kernel.
>>
>> [YOCTO #12858]
>>
>> Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
>> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
>> ---
>>   meta/classes/devtool-source.bbclass | 12 ++++++++++++
>>   1 file changed, 12 insertions(+)
>>
>> diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
>> index 56882a4..c70fea2 100644
>> --- a/meta/classes/devtool-source.bbclass
>> +++ b/meta/classes/devtool-source.bbclass
>> @@ -90,11 +90,23 @@ python devtool_post_unpack() {
>>                           fname in files])
>>           return ret
>>   
>> +    is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d)
>>       # Move local source files into separate subdir
>>       recipe_patches = [os.path.basename(patch) for patch in
>>                           oe.recipeutils.get_recipe_patches(d)]
>>       local_files = oe.recipeutils.get_recipe_local_files(d)
>>   
>> +    if is_kernel_yocto:
>> +      for key in local_files.copy():
>> +        if key.endswith('scc'):
>> +          sccfile = open(local_files[key], 'r')
>> +          for l in sccfile:
>> +            line = l.split()
>> +            if line and line[0] == 'kconf' and line[-1].endswith('.cfg'):
>> +              local_files[line[-1]] = os.path.join(os.path.dirname(local_files[key]), line[-1])
>> +              shutil.copy2(os.path.join(os.path.dirname(local_files[key]), line[-1]), workdir)
>> +          sccfile.close()
>> +
> Would the patches included in these .scc files also need to be handled
> in the same way? Would this also work if there are other scc files
> included in a scc file?
Yes, I believe that the same mechanism should be used for patch files as 
well,
basically anything that may be needed to build but that its not necessarily
explicitly listed on SRC_URI.

I believe it will work for other scc files and it doesnt have to be 
recursive,
because while cfg files arent required to be in SRC_URI , scc files ARE 
required
to be SRC_URI, which means that this will eventually find the other scc 
file on the list

Cheers,

Alejandro

>
> Thanks,
>
> Anuj

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Anuj Mittal]

On 08/02/2018 08:27 AM, Jaewon Lee wrote:
> Hi Anuj,
> 
> Thanks for the reply
> That is a good point, will send a v2 that also deals with patches.
> Regarding other scc files,  according to the kernel dev manual, 
> "It is only necessary to specify the .scc files on the SRC_URI."
> 
> So from my understanding all scc files will be in SRC_URI which means they get parsed by this fix, 
> Is this not the case?
If a scc file, foo.scc, includes any other scc file say bar.scc, then it
isn't mandatory to specify bar.scc as part of SRC_URI I think.

You can give it a try by creating empty sccs and including a malformed
patch in a nested scc to see if you get an error while patching.

Thanks,

Anuj

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Anuj Mittal]

On 08/02/2018 07:30 AM, Alejandro Enedino Hernandez Samaniego wrote:
> Hey Anuj,
> 
> 
> On 08/01/2018 04:25 AM, Anuj Mittal wrote:
>> On 07/31/2018 05:21 AM, Jaewon Lee wrote:
>>> When using a recipe space kernel-meta, scc files are added through
>>> SRC_URI, but they may include corresponding kernel fragments that are
>>> not necessarily in SRC_URI.
>>>
>>> For bitbake, this is not a problem because the kernel-yocto class adds
>>> the path where the .scc file was found to includes which consequentially
>>> makes the .cfg file available to the kernel build.
>>>
>>> However, when using devtool, only files specified in SRC_URI are copied
>>> to oe-local-files in devtool's workspace. So if the cfg file is not in
>>> SRC_URI, it won't be copied, causing a kernel build failure when trying
>>> to find it.
>>>
>>> This fix parses local .scc files in SRC_URI, copies the corresponding .cfg
>>> file to devtool's workdir, and also adds it to local_files so it is
>>> available when doing a devtool build for the kernel.
>>>
>>> [YOCTO #12858]
>>>
>>> Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
>>> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
>>> ---
>>>   meta/classes/devtool-source.bbclass | 12 ++++++++++++
>>>   1 file changed, 12 insertions(+)
>>>
>>> diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
>>> index 56882a4..c70fea2 100644
>>> --- a/meta/classes/devtool-source.bbclass
>>> +++ b/meta/classes/devtool-source.bbclass
>>> @@ -90,11 +90,23 @@ python devtool_post_unpack() {
>>>                           fname in files])
>>>           return ret
>>>   
>>> +    is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d)
>>>       # Move local source files into separate subdir
>>>       recipe_patches = [os.path.basename(patch) for patch in
>>>                           oe.recipeutils.get_recipe_patches(d)]
>>>       local_files = oe.recipeutils.get_recipe_local_files(d)
>>>   
>>> +    if is_kernel_yocto:
>>> +      for key in local_files.copy():
>>> +        if key.endswith('scc'):
>>> +          sccfile = open(local_files[key], 'r')
>>> +          for l in sccfile:
>>> +            line = l.split()
>>> +            if line and line[0] == 'kconf' and line[-1].endswith('.cfg'):
>>> +              local_files[line[-1]] = os.path.join(os.path.dirname(local_files[key]), line[-1])
>>> +              shutil.copy2(os.path.join(os.path.dirname(local_files[key]), line[-1]), workdir)
>>> +          sccfile.close()
>>> +
>> Would the patches included in these .scc files also need to be handled
>> in the same way? Would this also work if there are other scc files
>> included in a scc file?
> Yes, I believe that the same mechanism should be used for patch files as 
> well,
> basically anything that may be needed to build but that its not necessarily
> explicitly listed on SRC_URI.
> 
> I believe it will work for other scc files and it doesnt have to be 
> recursive,
> because while cfg files arent required to be in SRC_URI , scc files ARE 
> required
> to be SRC_URI, which means that this will eventually find the other scc 
> file on the list

I don't think they are required to be specified except for the top level
one. At least, when I test it, I see problems. kernel-tools spp script
parses them recursively and looks for a nested scc even if it is not
specified as part of SRC_URI. That is how the top level sccs from
kernel-cache are parsed too. Can you give it a try please?

Thanks,

Anuj

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Richard Purdie]

On Thu, 2018-08-02 at 10:55 +0800, Anuj Mittal wrote:
> On 08/02/2018 07:30 AM, Alejandro Enedino Hernandez Samaniego wrote:
> > Hey Anuj,
> > 
> > 
> > On 08/01/2018 04:25 AM, Anuj Mittal wrote:
> > > On 07/31/2018 05:21 AM, Jaewon Lee wrote:
> > > > When using a recipe space kernel-meta, scc files are added
> > > > through
> > > > SRC_URI, but they may include corresponding kernel fragments
> > > > that are
> > > > not necessarily in SRC_URI.
> > > > 
> > > > For bitbake, this is not a problem because the kernel-yocto
> > > > class adds
> > > > the path where the .scc file was found to includes which
> > > > consequentially
> > > > makes the .cfg file available to the kernel build.
> > > > 
> > > > However, when using devtool, only files specified in SRC_URI
> > > > are copied
> > > > to oe-local-files in devtool's workspace. So if the cfg file is
> > > > not in
> > > > SRC_URI, it won't be copied, causing a kernel build failure
> > > > when trying
> > > > to find it.
> > > > 
> > > > This fix parses local .scc files in SRC_URI, copies the
> > > > corresponding .cfg
> > > > file to devtool's workdir, and also adds it to local_files so
> > > > it is
> > > > available when doing a devtool build for the kernel.
> > > > 
> > > > [YOCTO #12858]
> > > > 
> > > > Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
> > > > Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@
> > > > xilinx.com>
> > > > ---
> > > >   meta/classes/devtool-source.bbclass | 12 ++++++++++++
> > > >   1 file changed, 12 insertions(+)
> > > > 
> > > > diff --git a/meta/classes/devtool-source.bbclass
> > > > b/meta/classes/devtool-source.bbclass
> > > > index 56882a4..c70fea2 100644
> > > > --- a/meta/classes/devtool-source.bbclass
> > > > +++ b/meta/classes/devtool-source.bbclass
> > > > @@ -90,11 +90,23 @@ python devtool_post_unpack() {
> > > >                           fname in files])
> > > >           return ret
> > > >   
> > > > +    is_kernel_yocto = bb.data.inherits_class('kernel-yocto',
> > > > d)
> > > >       # Move local source files into separate subdir
> > > >       recipe_patches = [os.path.basename(patch) for patch in
> > > >                           oe.recipeutils.get_recipe_patches(d)]
> > > >       local_files = oe.recipeutils.get_recipe_local_files(d)
> > > >   
> > > > +    if is_kernel_yocto:
> > > > +      for key in local_files.copy():
> > > > +        if key.endswith('scc'):
> > > > +          sccfile = open(local_files[key], 'r')
> > > > +          for l in sccfile:
> > > > +            line = l.split()
> > > > +            if line and line[0] == 'kconf' and line[-
> > > > 1].endswith('.cfg'):
> > > > +              local_files[line[-1]] =
> > > > os.path.join(os.path.dirname(local_files[key]), line[-1])
> > > > +              shutil.copy2(os.path.join(os.path.dirname(local_
> > > > files[key]), line[-1]), workdir)
> > > > +          sccfile.close()
> > > > +
> > > 
> > > Would the patches included in these .scc files also need to be
> > > handled
> > > in the same way? Would this also work if there are other scc
> > > files
> > > included in a scc file?
> > 
> > Yes, I believe that the same mechanism should be used for patch
> > files as 
> > well,
> > basically anything that may be needed to build but that its not
> > necessarily
> > explicitly listed on SRC_URI.
> > 
> > I believe it will work for other scc files and it doesnt have to
> > be 
> > recursive,
> > because while cfg files arent required to be in SRC_URI , scc files
> > ARE 
> > required
> > to be SRC_URI, which means that this will eventually find the other
> > scc 
> > file on the list
> 
> I don't think they are required to be specified except for the top
> level
> one. At least, when I test it, I see problems. kernel-tools spp
> script
> parses them recursively and looks for a nested scc even if it is not
> specified as part of SRC_URI. That is how the top level sccs from
> kernel-cache are parsed too. Can you give it a try please?

What would be really great here would be some test cases in the qa
framework! It'd be a good way to both illustrate the problems and then
test we've fixed it and that it stays fixed.

Cheers,

Richard

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Alejandro Enedino Hernandez Samaniego]

[-- Attachment #1: Type: text/plain, Size: 5522 bytes --]



On 08/02/2018 01:53 AM, Richard Purdie wrote:
> On Thu, 2018-08-02 at 10:55 +0800, Anuj Mittal wrote:
>> On 08/02/2018 07:30 AM, Alejandro Enedino Hernandez Samaniego wrote:
>>> Hey Anuj,
>>>
>>>
>>> On 08/01/2018 04:25 AM, Anuj Mittal wrote:
>>>> On 07/31/2018 05:21 AM, Jaewon Lee wrote:
>>>>> When using a recipe space kernel-meta, scc files are added
>>>>> through
>>>>> SRC_URI, but they may include corresponding kernel fragments
>>>>> that are
>>>>> not necessarily in SRC_URI.
>>>>>
>>>>> For bitbake, this is not a problem because the kernel-yocto
>>>>> class adds
>>>>> the path where the .scc file was found to includes which
>>>>> consequentially
>>>>> makes the .cfg file available to the kernel build.
>>>>>
>>>>> However, when using devtool, only files specified in SRC_URI
>>>>> are copied
>>>>> to oe-local-files in devtool's workspace. So if the cfg file is
>>>>> not in
>>>>> SRC_URI, it won't be copied, causing a kernel build failure
>>>>> when trying
>>>>> to find it.
>>>>>
>>>>> This fix parses local .scc files in SRC_URI, copies the
>>>>> corresponding .cfg
>>>>> file to devtool's workdir, and also adds it to local_files so
>>>>> it is
>>>>> available when doing a devtool build for the kernel.
>>>>>
>>>>> [YOCTO #12858]
>>>>>
>>>>> Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
>>>>> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@
>>>>> xilinx.com>
>>>>> ---
>>>>>    meta/classes/devtool-source.bbclass | 12 ++++++++++++
>>>>>    1 file changed, 12 insertions(+)
>>>>>
>>>>> diff --git a/meta/classes/devtool-source.bbclass
>>>>> b/meta/classes/devtool-source.bbclass
>>>>> index 56882a4..c70fea2 100644
>>>>> --- a/meta/classes/devtool-source.bbclass
>>>>> +++ b/meta/classes/devtool-source.bbclass
>>>>> @@ -90,11 +90,23 @@ python devtool_post_unpack() {
>>>>>                            fname in files])
>>>>>            return ret
>>>>>    
>>>>> +    is_kernel_yocto = bb.data.inherits_class('kernel-yocto',
>>>>> d)
>>>>>        # Move local source files into separate subdir
>>>>>        recipe_patches = [os.path.basename(patch) for patch in
>>>>>                            oe.recipeutils.get_recipe_patches(d)]
>>>>>        local_files = oe.recipeutils.get_recipe_local_files(d)
>>>>>    
>>>>> +    if is_kernel_yocto:
>>>>> +      for key in local_files.copy():
>>>>> +        if key.endswith('scc'):
>>>>> +          sccfile = open(local_files[key], 'r')
>>>>> +          for l in sccfile:
>>>>> +            line = l.split()
>>>>> +            if line and line[0] == 'kconf' and line[-
>>>>> 1].endswith('.cfg'):
>>>>> +              local_files[line[-1]] =
>>>>> os.path.join(os.path.dirname(local_files[key]), line[-1])
>>>>> +              shutil.copy2(os.path.join(os.path.dirname(local_
>>>>> files[key]), line[-1]), workdir)
>>>>> +          sccfile.close()
>>>>> +
>>>> Would the patches included in these .scc files also need to be
>>>> handled
>>>> in the same way? Would this also work if there are other scc
>>>> files
>>>> included in a scc file?
>>> Yes, I believe that the same mechanism should be used for patch
>>> files as
>>> well,
>>> basically anything that may be needed to build but that its not
>>> necessarily
>>> explicitly listed on SRC_URI.
>>>
>>> I believe it will work for other scc files and it doesnt have to
>>> be
>>> recursive,
>>> because while cfg files arent required to be in SRC_URI , scc files
>>> ARE
>>> required
>>> to be SRC_URI, which means that this will eventually find the other
>>> scc
>>> file on the list
>> I don't think they are required to be specified except for the top
>> level
>> one. At least, when I test it, I see problems. kernel-tools spp
>> script
>> parses them recursively and looks for a nested scc even if it is not
>> specified as part of SRC_URI. That is how the top level sccs from
>> kernel-cache are parsed too. Can you give it a try please?
Hey Anuj,
We can give it a try but do you have a specific example of how it fails 
for you?
Because I know theres lots of nested sccs on the yocto kernel cache, but 
in that case
it wouldn't be a problem since this bug is specifically introduced by 
devtool when it
copies local files from SRC_URI to a oe-local-files directory, it misses 
the corresponding cfg files (or patch files)
since they're not listed in SRC_URI and it fails to build, in the case 
of the yocto kernel cache, it
does not contain local files, so they wont be moved hence why it 
wouldn't be an issue.

 From the kernel dev manual:
https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#recipe-space-metadata 
:
It is only necessary to specify the|.scc|files on the|SRC_URI| 
<http://www.yoctoproject.org/docs/2.5/ref-manual/ref-manual.html#var-SRC_URI>.

It specifies that only the scc files need to be on SRC_URI, which is why 
I'm saying the script
will eventually run into them from the list.

I want to be clear that I'm not against doing this recursively, I just 
want to understand your testcase better.
> What would be really great here would be some test cases in the qa
> framework! It'd be a good way to both illustrate the problems and then
> test we've fixed it and that it stays fixed.
Hey Richard,
Sure that sounds like a good idea, well work on adding a testcase (or 
adding to the existing one) for devtool when
it is using a recipe-space kernel -meta.

Alejandro
>
> Cheers,
>
> Richard
>


[-- Attachment #2: Type: text/html, Size: 10319 bytes --]

Re: [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Anuj Mittal]

On 08/03/2018 04:55 AM, Alejandro Enedino Hernandez Samaniego wrote:
> We can give it a try but do you have a specific example of how it fails
> for you?
> Because I know theres lots of nested sccs on the yocto kernel cache, but
> in that case
> it wouldn't be a problem since this bug is specifically introduced by
> devtool when it
> copies local files from SRC_URI to a oe-local-files directory, it misses
> the corresponding cfg files (or patch files)
> since they're not listed in SRC_URI and it fails to build, in the case
> of the yocto kernel cache, it
> does not contain local files, so they wont be moved hence why it
> wouldn't be an issue.
> 
> From the kernel dev manual:
> https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#recipe-space-metadata
> :
> It is only necessary to specify the |.scc| files on the |SRC_URI|
> <http://www.yoctoproject.org/docs/2.5/ref-manual/ref-manual.html#var-SRC_URI>.

You are ignoring the line right next to it: "BitBake parses them and
fetches any files referenced in the .scc files by the include, patch, or
kconf commands." All files specified using 'include' will be parsed too
even if they are not in SRC_URI and this includes other scc files.

This change is good because it fixes something that is definitely
broken. My suggestion was meant to improve it further. I had provided
the test case in my reply to original patch.

And, I just tested it again using this change applied and I do see
errors when using a nested scc. Here it is again:

│   ├── linux
│   │   ├── linux-intel
│   │   │   ├── abc.scc
│   │   │   ├── def.scc

abc.scc only includes def.scc. And, def.scc includes a patch. SRC_URI
includes only abc.scc.

> 
> It specifies that only the scc files need to be on SRC_URI, which is why
> I'm saying the script
> will eventually run into them from the list.
> 
> I want to be clear that I'm not against doing this recursively, I just
> want to understand your testcase better.

I think if you'd not like to consider this use case as part of this
change, that should also be fine. But, do consider adding a new bug in
that case.