[PATCH] cooker.py: multiconfig support for findBestProvider

[PATCH] cooker.py: multiconfig support for findBestProvider

[Juro Bystricky]

In a multiconfig environment, a tinfoil call such as

    tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc")

can fail with an error such as:

  File "/data/master/poky/bitbake/lib/bb/tinfoil.py", line 373, in get_recipe_file
    raise bb.providers.NoProvider('Unable to find any recipe file matching "%s"' % pn)
bb.providers.NoProvider: Unable to find any recipe file matching "multiconfig:arduino-101-sss:gcc"

The culprit is findBestProvider (only called from tinfoil), which does not
handle multiconfig. This patch fixes the error and the tinfoil call returns absolute path
to the recipe, i.e:

  "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"

[YOCTO#11210]

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
---
 bitbake/lib/bb/cooker.py | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index bc8574a..bf9593e 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -160,6 +160,19 @@ class EventWriter:
 #============================================================================#
 # BBCooker
 #============================================================================#
+
+def split_mc_pn(pn):
+    if pn.startswith("multiconfig:"):
+        mc = pn.split(":")[1]
+        pn = ":".join(pn.split(":")[2:])
+        return (mc, pn)
+    return ('', pn)
+
+def split_pn(pn):
+    if pn.startswith("multiconfig:"):
+       return ":".join(pn.split(":")[2:])
+    return pn
+
 class BBCooker:
     """
     Manages one bitbake build run
@@ -1104,11 +1117,12 @@ class BBCooker:
     def findProviders(self, mc=''):
         return bb.providers.findProviders(self.data, self.recipecaches[mc], self.recipecaches[mc].pkg_pn)
 
-    def findBestProvider(self, pn, mc=''):
+    def findBestProvider(self, pn):
+        (mc, pn) = split_mc_pn(pn)
         if pn in self.recipecaches[mc].providers:
             filenames = self.recipecaches[mc].providers[pn]
             eligible, foundUnique = bb.providers.filterProviders(filenames, pn, self.data, self.recipecaches[mc])
-            filename = eligible[0]
+            filename = split_pn(eligible[0])
             return None, None, None, filename
         elif pn in self.recipecaches[mc].pkg_pn:
             return bb.providers.findBestProvider(pn, self.data, self.recipecaches[mc], self.recipecaches[mc].pkg_pn)
-- 
2.7.4

Re: [PATCH] cooker.py: multiconfig support for findBestProvider

[Patrick Ohly]

On Wed, 2017-04-05 at 14:34 -0700, Juro Bystricky wrote:
> In a multiconfig environment, a tinfoil call such as
> 
>     tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc")
> 
> can fail with an error such as:
> 
>   File "/data/master/poky/bitbake/lib/bb/tinfoil.py", line 373, in get_recipe_file
>     raise bb.providers.NoProvider('Unable to find any recipe file matching "%s"' % pn)
> bb.providers.NoProvider: Unable to find any recipe file matching "multiconfig:arduino-101-sss:gcc"
> 
> The culprit is findBestProvider (only called from tinfoil), which does not
> handle multiconfig. This patch fixes the error and the tinfoil call returns absolute path
> to the recipe, i.e:
> 
>   "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"

So tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc") and
tinfoil.parse_recipe("gcc") then return identical data, because both end
up parsing "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"
without taking the "arduino-101-sss" config into account?

That wouldn't be correct.

As far as I understand it, the mapping has to be:
multiconfig:arduino-101-sss:gcc ->
multiconfig:arduino-101-sss:/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH] cooker.py: multiconfig support for findBestProvider

[Richard Purdie]

On Thu, 2017-04-06 at 10:43 +0200, Patrick Ohly wrote:
> On Wed, 2017-04-05 at 14:34 -0700, Juro Bystricky wrote:
> > 
> > In a multiconfig environment, a tinfoil call such as
> > 
> >     tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc")
> > 
> > can fail with an error such as:
> > 
> >   File "/data/master/poky/bitbake/lib/bb/tinfoil.py", line 373, in
> > get_recipe_file
> >     raise bb.providers.NoProvider('Unable to find any recipe file
> > matching "%s"' % pn)
> > bb.providers.NoProvider: Unable to find any recipe file matching
> > "multiconfig:arduino-101-sss:gcc"
> > 
> > The culprit is findBestProvider (only called from tinfoil), which
> > does not
> > handle multiconfig. This patch fixes the error and the tinfoil call
> > returns absolute path
> > to the recipe, i.e:
> > 
> >   "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"
> So tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc") and
> tinfoil.parse_recipe("gcc") then return identical data, because both
> end
> up parsing "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"
> without taking the "arduino-101-sss" config into account?
> 
> That wouldn't be correct.
> 
> As far as I understand it, the mapping has to be:
> multiconfig:arduino-101-sss:gcc ->
> multiconfig:arduino-101-sss:/data/master/poky/meta/recipes-
> devtools/gcc/gcc_6.3.bb

I agree with Patrick. At a quick glance, I think the caller
of findBestProvider is going to need to handle the mc, which is
why findBestProvider takes it as a parameter.

Cheers,

Richard

Re: [PATCH] cooker.py: multiconfig support for findBestProvider

[Bystricky, Juro]

Thanks, v2 coming soon. This clarifies some other issues as well.

Juro
________________________________________
From: Richard Purdie [richard.purdie@linuxfoundation.org]
Sent: Thursday, April 06, 2017 4:45 AM
To: Ohly, Patrick; Bystricky, Juro
Cc: bitbake-devel@lists.openembedded.org
Subject: Re: [bitbake-devel] [PATCH] cooker.py: multiconfig support for findBestProvider

On Thu, 2017-04-06 at 10:43 +0200, Patrick Ohly wrote:
> On Wed, 2017-04-05 at 14:34 -0700, Juro Bystricky wrote:
> >
> > In a multiconfig environment, a tinfoil call such as
> >
> >     tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc")
> >
> > can fail with an error such as:
> >
> >   File "/data/master/poky/bitbake/lib/bb/tinfoil.py", line 373, in
> > get_recipe_file
> >     raise bb.providers.NoProvider('Unable to find any recipe file
> > matching "%s"' % pn)
> > bb.providers.NoProvider: Unable to find any recipe file matching
> > "multiconfig:arduino-101-sss:gcc"
> >
> > The culprit is findBestProvider (only called from tinfoil), which
> > does not
> > handle multiconfig. This patch fixes the error and the tinfoil call
> > returns absolute path
> > to the recipe, i.e:
> >
> >   "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"
> So tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc") and
> tinfoil.parse_recipe("gcc") then return identical data, because both
> end
> up parsing "/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"
> without taking the "arduino-101-sss" config into account?
>
> That wouldn't be correct.
>
> As far as I understand it, the mapping has to be:
> multiconfig:arduino-101-sss:gcc ->
> multiconfig:arduino-101-sss:/data/master/poky/meta/recipes-
> devtools/gcc/gcc_6.3.bb

I agree with Patrick. At a quick glance, I think the caller
of findBestProvider is going to need to handle the mc, which is
why findBestProvider takes it as a parameter.

Cheers,

Richard