Overriding fixmepath

Overriding fixmepath

[Kristian Amlie]

After the new recipe sysroots were introduced we've been having trouble
compiling Go. I've narrowed it down to the path replacement mechanism
that goes on using fixmepath.

For example, this does not work:

  bitbake -c cleansstate go-cross
  bitbake go-cross

But this does:

  bitbake -c cleansstate go-cross
  bitbake -c prepare_recipe_sysroot go-cross
  rm -rf
/home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3
  cp -r
/home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-components/x86_64/go-bootstrap-native/usr/lib/go-bootstrap-native-1.4.3
/home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3
  bitbake go-cross

IOW, if I replace the recipe sysroot with the originals, it works, but
if I keep the modified files put there by the prepare_recipe_sysroot
task, it doesn't.

And this is where I admittedly get a bit lost: I haven't fully
understood how the fixmepath mechanism operates, but from what I can
tell, this path replacement is not appropriate for Go. It doesn't care
where it's installed, and always operates relative to the GOROOT
environment variable, which is set by all Go recipes.

Why the path replacement triggers an error I cannot say, but I suspect
it has to do with the fact that Go object files are a bit special
compared to C object files, and this operation in fact corrupts them.
More mysterious still is why this happens only on some build machines,
and not others, but once triggered it appears to be consistently triggered.

Any advice for what to do next? I could try to skip the fixmepath
operation completely, but I'm not sure how, and it feels a bit like
using a sledgehammer where I should be using a scalpel.

Any help appreciated!

-- 
Kristian

Re: Overriding fixmepath

[Richard Purdie]

On Wed, 2017-01-25 at 11:43 +0100, Kristian Amlie wrote:
> After the new recipe sysroots were introduced we've been having
> trouble
> compiling Go. I've narrowed it down to the path replacement mechanism
> that goes on using fixmepath.
> 
> For example, this does not work:
> 
>   bitbake -c cleansstate go-cross
>   bitbake go-cross
> 
> But this does:
> 
>   bitbake -c cleansstate go-cross
>   bitbake -c prepare_recipe_sysroot go-cross
>   rm -rf
> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-
> linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-
> native-1.4.3
>   cp -r
> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
> components/x86_64/go-bootstrap-native/usr/lib/go-bootstrap-native-
> 1.4.3
> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-
> linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-
> native-1.4.3
>   bitbake go-cross
> 
> IOW, if I replace the recipe sysroot with the originals, it works,
> but
> if I keep the modified files put there by the prepare_recipe_sysroot
> task, it doesn't.
> 
> And this is where I admittedly get a bit lost: I haven't fully
> understood how the fixmepath mechanism operates, but from what I can
> tell, this path replacement is not appropriate for Go. It doesn't
> care
> where it's installed, and always operates relative to the GOROOT
> environment variable, which is set by all Go recipes.

Does this still work if you remove the
/home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
components/x86_64/go-bootstrap-native directory before building go-
cross? I worry that its actually referencing the other location and
using files from there (a different recipe's workdir) and that
therefore there are a different set of other problems this recipe also
has.

> Why the path replacement triggers an error I cannot say, but I
> suspect it has to do with the fact that Go object files are a bit
> special compared to C object files, and this operation in fact
> corrupts them. More mysterious still is why this happens only on some
> build machines, and not others, but once triggered it appears to be
> consistently triggered.

The fixmepath replacements only happen on text files. It could be some
relocation is needed on the binaries too, perhaps through configuration
on the commandline or perhaps through the environment. We have had to
patch some tools to allow relocation in the past too.

> Any advice for what to do next? I could try to skip the fixmepath
> operation completely, but I'm not sure how, and it feels a bit like
> using a sledgehammer where I should be using a scalpel.

Try removing the path I suggested after copying and see if it still
works. If it does, I'd probably try and find which subset of the files
breaks it, narrow down the issue. If removing it breaks things, there
is a whole world of bigger issues :(

Its possible to add specific inclusions/exclusions from fixmepath FWIW
but we need to understand more about what is happening first.

Cheers,

Richard

Re: Overriding fixmepath

[Kristian Amlie]

[-- Attachment #1: Type: text/plain, Size: 3931 bytes --]

On 25/01/17 12:24, Richard Purdie wrote:
> On Wed, 2017-01-25 at 11:43 +0100, Kristian Amlie wrote:
>> After the new recipe sysroots were introduced we've been having
>> trouble
>> compiling Go. I've narrowed it down to the path replacement mechanism
>> that goes on using fixmepath.
>>
>> For example, this does not work:
>>
>>   bitbake -c cleansstate go-cross
>>   bitbake go-cross
>>
>> But this does:
>>
>>   bitbake -c cleansstate go-cross
>>   bitbake -c prepare_recipe_sysroot go-cross
>>   rm -rf
>> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-
>> linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-
>> native-1.4.3
>>   cp -r
>> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
>> components/x86_64/go-bootstrap-native/usr/lib/go-bootstrap-native-
>> 1.4.3
>> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-
>> linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-
>> native-1.4.3
>>   bitbake go-cross
>>
>> IOW, if I replace the recipe sysroot with the originals, it works,
>> but
>> if I keep the modified files put there by the prepare_recipe_sysroot
>> task, it doesn't.
>>
>> And this is where I admittedly get a bit lost: I haven't fully
>> understood how the fixmepath mechanism operates, but from what I can
>> tell, this path replacement is not appropriate for Go. It doesn't
>> care
>> where it's installed, and always operates relative to the GOROOT
>> environment variable, which is set by all Go recipes.
> 
> Does this still work if you remove the
> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
> components/x86_64/go-bootstrap-native directory before building go-
> cross? I worry that its actually referencing the other location and
> using files from there (a different recipe's workdir) and that
> therefore there are a different set of other problems this recipe also
> has.

You mean after executing prepare_recipe_sysroot, right? I tried this,
but it made no difference to my previous results, so presumably it is
looking in the right place.

> 
>> Why the path replacement triggers an error I cannot say, but I
>> suspect it has to do with the fact that Go object files are a bit
>> special compared to C object files, and this operation in fact
>> corrupts them. More mysterious still is why this happens only on some
>> build machines, and not others, but once triggered it appears to be
>> consistently triggered.
> 
> The fixmepath replacements only happen on text files. It could be some
> relocation is needed on the binaries too, perhaps through configuration
> on the commandline or perhaps through the environment. We have had to
> patch some tools to allow relocation in the past too.

I see this happening to .a files as well. Aren't they normally object
files? I have attached the fixmepath file for go-bootstrap-native, which
go-cross uses to build itself.

Looking at one of the files from there, it has quite peculiar contents,
probably something Go internal which we shouldn't mess with:

  $ ar t template.a
  __.PKGDEF
  _go_.6

>> Any advice for what to do next? I could try to skip the fixmepath
>> operation completely, but I'm not sure how, and it feels a bit like
>> using a sledgehammer where I should be using a scalpel.
> 
> Try removing the path I suggested after copying and see if it still
> works. If it does, I'd probably try and find which subset of the files
> breaks it, narrow down the issue. If removing it breaks things, there
> is a whole world of bigger issues :(
> 
> Its possible to add specific inclusions/exclusions from fixmepath FWIW
> but we need to understand more about what is happening first.

Take a look at the attached file, to me the inclusion of .a files in the
fixmepath file seems wrong, so I can start with that, unless you have
better idea.

-- 
Kristian

[-- Attachment #2: fixmepath.txt --]
[-- Type: text/plain, Size: 1653 bytes --]

recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/html/template.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/syscall.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/crypto/x509.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/crypto/tls.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/text/template.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/go/ast.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/debug/elf.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/cgi.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/httputil.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/fcgi.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/httptest.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/pprof.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/smtp.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/rpc.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/cmd/pprof/internal/driver.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/cmd/internal/rsc.io/arm/armasm.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/src/runtime/zversion.go

Re: Overriding fixmepath

[Richard Purdie]

On Wed, 2017-01-25 at 13:22 +0100, Kristian Amlie wrote:
> On 25/01/17 12:24, Richard Purdie wrote:
> > 
> > On Wed, 2017-01-25 at 11:43 +0100, Kristian Amlie wrote:
> > Does this still work if you remove the
> > /home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
> > components/x86_64/go-bootstrap-native directory before building go-
> > cross? I worry that its actually referencing the other location and
> > using files from there (a different recipe's workdir) and that
> > therefore there are a different set of other problems this recipe
> > also
> > has.
> You mean after executing prepare_recipe_sysroot, right? I tried this,
> but it made no difference to my previous results, so presumably it is
> looking in the right place.

Ok, that is good and probably means this should be easier to fix.

> > > Why the path replacement triggers an error I cannot say, but I
> > > suspect it has to do with the fact that Go object files are a bit
> > > special compared to C object files, and this operation in fact
> > > corrupts them. More mysterious still is why this happens only on
> > > some
> > > build machines, and not others, but once triggered it appears to
> > > be
> > > consistently triggered.
> > The fixmepath replacements only happen on text files. It could be
> > some
> > relocation is needed on the binaries too, perhaps through
> > configuration
> > on the commandline or perhaps through the environment. We have had
> > to
> > patch some tools to allow relocation in the past too.
> I see this happening to .a files as well. Aren't they normally object
> files? I have attached the fixmepath file for go-bootstrap-native,
> which
> go-cross uses to build itself.
> 
> Looking at one of the files from there, it has quite peculiar
> contents,
> probably something Go internal which we shouldn't mess with:
> 
>   $ ar t template.a
>   __.PKGDEF
>   _go_.6
> 
> > 
> > > 
> > > Any advice for what to do next? I could try to skip the fixmepath
> > > operation completely, but I'm not sure how, and it feels a bit
> > > like
> > > using a sledgehammer where I should be using a scalpel.
> > Try removing the path I suggested after copying and see if it still
> > works. If it does, I'd probably try and find which subset of the
> > files
> > breaks it, narrow down the issue. If removing it breaks things,
> > there
> > is a whole world of bigger issues :(
> > 
> > Its possible to add specific inclusions/exclusions from fixmepath
> > FWIW
> > but we need to understand more about what is happening first.
> Take a look at the attached file, to me the inclusion of .a files in
> the
> fixmepath file seems wrong, so I can start with that, unless you have
> better idea.

I think this is a very good lead. The code searches for replacements to
make with: "grep -Irl" (see sstate.bbclass) and it looks like its
matching these files and shouldn't. We may need to make sstate.bbclass
look something like:

SSTATE_SCAN_CMD ?= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
SSTATE_SCAN_CMD_NATIVE ?= 'grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} ${SSTATE_BUILDDIR}'

and then:
    if bb.data.inherits_class('native', d) or bb.data.inherits_class('cross', d) or bb.data.inherits_class('sdk', d) or bb.data.inherits_class('crosssdk', d):
        d.setVar('SSTATE_SCAN_CMD', '${SSTATE_SCAN_CMD_NATIVE}')

which means you'd be able to override the command being used to exclude
.a files?

Cheers,

Richard

Re: Overriding fixmepath

[Kristian Amlie]

>> Take a look at the attached file, to me the inclusion of .a files in
>> the
>> fixmepath file seems wrong, so I can start with that, unless you have
>> better idea.
> 
> I think this is a very good lead. The code searches for replacements to
> make with: "grep -Irl" (see sstate.bbclass) and it looks like its
> matching these files and shouldn't. We may need to make sstate.bbclass
> look something like:
> 
> SSTATE_SCAN_CMD ?= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
> SSTATE_SCAN_CMD_NATIVE ?= 'grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} ${SSTATE_BUILDDIR}'
> 
> and then:
>     if bb.data.inherits_class('native', d) or bb.data.inherits_class('cross', d) or bb.data.inherits_class('sdk', d) or bb.data.inherits_class('crosssdk', d):
>         d.setVar('SSTATE_SCAN_CMD', '${SSTATE_SCAN_CMD_NATIVE}')
> 
> which means you'd be able to override the command being used to exclude
> .a files?

Thanks, good suggestion!

In order to make this overridable from any recipe, may I propose the following
patch instead? It has the advantage of avoiding the Python logic altogether, and
using weak defaults so that it can be easily overridden. One thing to note is
that I couldn't find the "sdk" class that was referenced in the Python
code. Does it exist anymore?

The mentioned patch is what I used, and it worked for me when I also added:

  SSTATE_SCAN_CMD = "true"

to the Go recipe.

-- 
Kristian

[PATCH] Make SSTATE_SCAN_CMD vars configurable using weak defaults.

[Kristian Amlie]

For this we move them out of the python section so they can be
overridden on a per-recipe basis.

The motivation for this change is that not all tool chains need the
path modifications provided by the command, and these will provide
alternative or empty commands. The Go compiler is such an example.

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
---
 meta/classes/cross.bbclass    | 4 ++++
 meta/classes/crosssdk.bbclass | 4 ++++
 meta/classes/native.bbclass   | 4 ++++
 meta/classes/sstate.bbclass   | 9 ++-------
 4 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/meta/classes/cross.bbclass b/meta/classes/cross.bbclass
index d125f10..4985f89 100644
--- a/meta/classes/cross.bbclass
+++ b/meta/classes/cross.bbclass
@@ -38,6 +38,10 @@ TOOLCHAIN_OPTIONS = ""
 
 DEPENDS_GETTEXT = "gettext-native"
 
+# This class encodes staging paths into its scripts data so can only be
+# reused if we manipulate the paths.
+SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
+
 # Path mangling needed by the cross packaging
 # Note that we use := here to ensure that libdir and includedir are
 # target paths.
diff --git a/meta/classes/crosssdk.bbclass b/meta/classes/crosssdk.bbclass
index c5c29ea..a5ffd97 100644
--- a/meta/classes/crosssdk.bbclass
+++ b/meta/classes/crosssdk.bbclass
@@ -10,6 +10,10 @@ python () {
 
 STAGING_BINDIR_TOOLCHAIN = "${STAGING_DIR_NATIVE}${bindir_native}/${TARGET_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
 
+# This class encodes staging paths into its scripts data so can only be
+# reused if we manipulate the paths.
+SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
+
 TARGET_ARCH = "${SDK_ARCH}"
 TARGET_VENDOR = "${SDK_VENDOR}"
 TARGET_OS = "${SDK_OS}"
diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
index 4d4aaad..ec91fc3 100644
--- a/meta/classes/native.bbclass
+++ b/meta/classes/native.bbclass
@@ -115,6 +115,10 @@ MACHINEOVERRIDES = ""
 
 PATH_prepend = "${COREBASE}/scripts/native-intercept:"
 
+# This class encodes staging paths into its scripts data so can only be
+# reused if we manipulate the paths.
+SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
+
 python native_virtclass_handler () {
     classextend = e.data.getVar('BBCLASSEXTEND') or ""
     if "native" not in classextend:
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index a767a02..5f4ef76 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -34,7 +34,8 @@ SSTATE_DUPWHITELIST += "${DEPLOY_DIR_SRC}"
 SSTATE_DUPWHITELIST += "${DEPLOY_DIR}/sdk/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt"
 
 SSTATE_SCAN_FILES ?= "*.la *-config *_config"
-SSTATE_SCAN_CMD ?= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
+SSTATE_SCAN_CMD ??= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
+SSTATE_SCAN_CMD_NATIVE ??= 'grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} ${SSTATE_BUILDDIR}'
 
 BB_HASHFILENAME = "False ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}"
 
@@ -94,12 +95,6 @@ python () {
         d.setVar('BB_HASHFILENAME', "True ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}")
         d.setVar('SSTATE_EXTRAPATHWILDCARD', "*/")
 
-    # These classes encode staging paths into their scripts data so can only be
-    # reused if we manipulate the paths
-    if bb.data.inherits_class('native', d) or bb.data.inherits_class('cross', d) or bb.data.inherits_class('sdk', d) or bb.data.inherits_class('crosssdk', d):
-        scan_cmd = "grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} ${SSTATE_BUILDDIR}"
-        d.setVar('SSTATE_SCAN_CMD', scan_cmd)
-
     unique_tasks = sorted(set((d.getVar('SSTATETASKS') or "").split()))
     d.setVar('SSTATETASKS', " ".join(unique_tasks))
     for task in unique_tasks:
-- 
2.7.4

✗ patchtest: failure for Make SSTATE_SCAN_CMD vars configurable using weak defaults.

[Patchwork]

== Series Details ==

Series: Make SSTATE_SCAN_CMD vars configurable using weak defaults.
Revision: 1
URL   : https://patchwork.openembedded.org/series/4967/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch            Make SSTATE_SCAN_CMD vars configurable using weak defaults.
 Issue             Shortlog does not follow expected format [test_shortlog_format] 
  Suggested fix    Commit shortlog (first line of commit message) should follow the format "<target>: <summary>"



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: ✗ patchtest: failure for Make SSTATE_SCAN_CMD vars configurable using weak defaults.

[Kristian Amlie]

Sorry I forgot the right patch format, but let me know if you agree with
the patch first, and I'll repost.

-- 
Kristian

On 25/01/17 15:23, Patchwork wrote:
> == Series Details ==
> 
> Series: Make SSTATE_SCAN_CMD vars configurable using weak defaults.
> Revision: 1
> URL   : https://patchwork.openembedded.org/series/4967/
> State : failure
> 
> == Summary ==
> 
> 
> Thank you for submitting this patch series to OpenEmbedded Core. This is
> an automated response. Several tests have been executed on the proposed
> series by patchtest resulting in the following failures:
> 
> 
> 
> * Patch            Make SSTATE_SCAN_CMD vars configurable using weak defaults.
>  Issue             Shortlog does not follow expected format [test_shortlog_format] 
>   Suggested fix    Commit shortlog (first line of commit message) should follow the format "<target>: <summary>"
> 
> 
> 
> If you believe any of these test results are incorrect, please reply to the
> mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> Otherwise we would appreciate you correcting the issues and submitting a new
> version of the patchset if applicable. Please ensure you add/increment the
> version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> [PATCH v3] -> ...).
> 
> ---
> Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
> 

Re: ✗ patchtest: failure for Make SSTATE_SCAN_CMD vars configurable using weak defaults.

[Richard Purdie]

On Wed, 2017-01-25 at 15:34 +0100, Kristian Amlie wrote:
> Sorry I forgot the right patch format, but let me know if you agree
> with the patch first, and I'll repost.

Yes, I'm ok with it. The sdk class is long gone.

Cheers,

Richard

[PATCH] sstate: Make SSTATE_SCAN_CMD vars configurable using weak defaults.

[Kristian Amlie]

For this we move them out of the python section so they can be
overridden on a per-recipe basis.

The motivation for this change is that not all tool chains need the
path modifications provided by the command, and these will provide
alternative or empty commands. The Go compiler is such an example.

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
---
 meta/classes/cross.bbclass    | 4 ++++
 meta/classes/crosssdk.bbclass | 4 ++++
 meta/classes/native.bbclass   | 4 ++++
 meta/classes/sstate.bbclass   | 9 ++-------
 4 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/meta/classes/cross.bbclass b/meta/classes/cross.bbclass
index d125f10..4985f89 100644
--- a/meta/classes/cross.bbclass
+++ b/meta/classes/cross.bbclass
@@ -38,6 +38,10 @@ TOOLCHAIN_OPTIONS = ""
 
 DEPENDS_GETTEXT = "gettext-native"
 
+# This class encodes staging paths into its scripts data so can only be
+# reused if we manipulate the paths.
+SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
+
 # Path mangling needed by the cross packaging
 # Note that we use := here to ensure that libdir and includedir are
 # target paths.
diff --git a/meta/classes/crosssdk.bbclass b/meta/classes/crosssdk.bbclass
index c5c29ea..a5ffd97 100644
--- a/meta/classes/crosssdk.bbclass
+++ b/meta/classes/crosssdk.bbclass
@@ -10,6 +10,10 @@ python () {
 
 STAGING_BINDIR_TOOLCHAIN = "${STAGING_DIR_NATIVE}${bindir_native}/${TARGET_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
 
+# This class encodes staging paths into its scripts data so can only be
+# reused if we manipulate the paths.
+SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
+
 TARGET_ARCH = "${SDK_ARCH}"
 TARGET_VENDOR = "${SDK_VENDOR}"
 TARGET_OS = "${SDK_OS}"
diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
index 4d4aaad..ec91fc3 100644
--- a/meta/classes/native.bbclass
+++ b/meta/classes/native.bbclass
@@ -115,6 +115,10 @@ MACHINEOVERRIDES = ""
 
 PATH_prepend = "${COREBASE}/scripts/native-intercept:"
 
+# This class encodes staging paths into its scripts data so can only be
+# reused if we manipulate the paths.
+SSTATE_SCAN_CMD ?= "${SSTATE_SCAN_CMD_NATIVE}"
+
 python native_virtclass_handler () {
     classextend = e.data.getVar('BBCLASSEXTEND') or ""
     if "native" not in classextend:
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index a767a02..5f4ef76 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -34,7 +34,8 @@ SSTATE_DUPWHITELIST += "${DEPLOY_DIR_SRC}"
 SSTATE_DUPWHITELIST += "${DEPLOY_DIR}/sdk/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt"
 
 SSTATE_SCAN_FILES ?= "*.la *-config *_config"
-SSTATE_SCAN_CMD ?= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
+SSTATE_SCAN_CMD ??= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
+SSTATE_SCAN_CMD_NATIVE ??= 'grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} ${SSTATE_BUILDDIR}'
 
 BB_HASHFILENAME = "False ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}"
 
@@ -94,12 +95,6 @@ python () {
         d.setVar('BB_HASHFILENAME', "True ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}")
         d.setVar('SSTATE_EXTRAPATHWILDCARD', "*/")
 
-    # These classes encode staging paths into their scripts data so can only be
-    # reused if we manipulate the paths
-    if bb.data.inherits_class('native', d) or bb.data.inherits_class('cross', d) or bb.data.inherits_class('sdk', d) or bb.data.inherits_class('crosssdk', d):
-        scan_cmd = "grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} ${SSTATE_BUILDDIR}"
-        d.setVar('SSTATE_SCAN_CMD', scan_cmd)
-
     unique_tasks = sorted(set((d.getVar('SSTATETASKS') or "").split()))
     d.setVar('SSTATETASKS', " ".join(unique_tasks))
     for task in unique_tasks:
-- 
2.7.4