sstate-cache and native recipes

sstate-cache and native recipes

[Gary Thomas]

I've been trying to understand (as was recently posted here by
another user) why native recipes are not being well shared/reused
by the sstate-cache mechanism.  I have a build machine where I
do lots of builds for various targets.  I would think (hope!)
that xxx-native packages would be the same for every build and
if I have the sstate mirror set up correctly, be shared appropriately.
To this end, I have one "master" build that I always run first
whenever the metadata changes, then point all the other builds
at that tree.

For the most part, this works pretty well, but there are still
some places where it doesn't.  Here are a couple of examples
which I've investigated:

* glib-2.0-native depends on ${DISTRO_FEATURES}
   To me this seems silly as "native" should be "native" and
   not depend on any distribution settings.  Here's the code
   that's causing it (in do_install)

         if [ -f ${D}${datadir}/installed-tests/glib/gdbus-serialization.test ]; then
                 if ${@bb.utils.contains("DISTRO_FEATURES", "x11", "false", "true", d)}; then
                         rm ${D}${datadir}/installed-tests/glib/gdbus-serialization.test
                 fi
         fi

   Obviously this isn't important for a native package.  Any
   suggestions on how I might keep this from creeping in?

* xxx-native packages are not shared if ${DISTRO} is different.
   Again, this seems wrong as "native" packages should really only
   depend on the build ${HOST}, not ${DISTRO}

I'm looking at this for the sole purpose of reducing the load on my
build machine.  Any package that can be shared and not have to be
rebuilt just makes it more productive :-)

Thanks for any ideas or comments

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate-cache and native recipes

[Gary Thomas]

On 2017-01-18 10:13, Gary Thomas wrote:
> I've been trying to understand (as was recently posted here by
> another user) why native recipes are not being well shared/reused
> by the sstate-cache mechanism.  I have a build machine where I
> do lots of builds for various targets.  I would think (hope!)
> that xxx-native packages would be the same for every build and
> if I have the sstate mirror set up correctly, be shared appropriately.
> To this end, I have one "master" build that I always run first
> whenever the metadata changes, then point all the other builds
> at that tree.
>
> For the most part, this works pretty well, but there are still
> some places where it doesn't.  Here are a couple of examples
> which I've investigated:
>
> * glib-2.0-native depends on ${DISTRO_FEATURES}
>   To me this seems silly as "native" should be "native" and
>   not depend on any distribution settings.  Here's the code
>   that's causing it (in do_install)
>
>         if [ -f ${D}${datadir}/installed-tests/glib/gdbus-serialization.test ]; then
>                 if ${@bb.utils.contains("DISTRO_FEATURES", "x11", "false", "true", d)}; then
>                         rm ${D}${datadir}/installed-tests/glib/gdbus-serialization.test
>                 fi
>         fi
>
>   Obviously this isn't important for a native package.  Any
>   suggestions on how I might keep this from creeping in?

BTW, once glib-2.0-native can't be shared, a number of other native
packages will also have to be rebuilt because of build dependencies.
I discovered this when qemu-native was being rebuilt on almost all
of my targets...

>
> * xxx-native packages are not shared if ${DISTRO} is different.
>   Again, this seems wrong as "native" packages should really only
>   depend on the build ${HOST}, not ${DISTRO}
>
> I'm looking at this for the sole purpose of reducing the load on my
> build machine.  Any package that can be shared and not have to be
> rebuilt just makes it more productive :-)
>
> Thanks for any ideas or comments
>


-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate-cache and native recipes

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

On 18 January 2017 at 09:13, Gary Thomas <gary@mlbassoc.com> wrote:

> * glib-2.0-native depends on ${DISTRO_FEATURES}
>   To me this seems silly as "native" should be "native" and
>   not depend on any distribution settings.  Here's the code
>   that's causing it (in do_install)
>
>         if [ -f ${D}${datadir}/installed-tests/glib/gdbus-serialization.test
> ]; then
>                 if ${@bb.utils.contains("DISTRO_FEATURES", "x11",
> "false", "true", d)}; then
>                         rm ${D}${datadir}/installed-tests
> /glib/gdbus-serialization.test
>                 fi
>         fi
>
>   Obviously this isn't important for a native package.  Any
>   suggestions on how I might keep this from creeping in?
>

The tests are only installed for target builds anyway, so that do_install
part could be target-specific.

Ross

[-- Attachment #2: Type: text/html, Size: 1401 bytes --]

Re: sstate-cache and native recipes

[Gary Thomas]

On 2017-01-18 10:35, Burton, Ross wrote:
>
> On 18 January 2017 at 09:13, Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>> wrote:
>
>     * glib-2.0-native depends on ${DISTRO_FEATURES}
>       To me this seems silly as "native" should be "native" and
>       not depend on any distribution settings.  Here's the code
>       that's causing it (in do_install)
>
>             if [ -f ${D}${datadir}/installed-tests/glib/gdbus-serialization.test ]; then
>                     if ${@bb.utils.contains("DISTRO_FEATURES", "x11", "false", "true", d)}; then
>                             rm ${D}${datadir}/installed-tests/glib/gdbus-serialization.test
>                     fi
>             fi
>
>       Obviously this isn't important for a native package.  Any
>       suggestions on how I might keep this from creeping in?
>
>
> The tests are only installed for target builds anyway, so that do_install part could be target-specific.

How would one change the recipe to reflect that?  Is there an
override that effectively says "not -native"?

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: sstate-cache and native recipes

[Jussi Kukkonen]

[-- Attachment #1: Type: text/plain, Size: 1263 bytes --]

On 18 January 2017 at 11:51, Gary Thomas <gary@mlbassoc.com> wrote:

> On 2017-01-18 10:35, Burton, Ross wrote:
>
>>
>> On 18 January 2017 at 09:13, Gary Thomas <gary@mlbassoc.com <mailto:
>> gary@mlbassoc.com>> wrote:
>>
>>     * glib-2.0-native depends on ${DISTRO_FEATURES}
>>       To me this seems silly as "native" should be "native" and
>>       not depend on any distribution settings.  Here's the code
>>       that's causing it (in do_install)
>>
>>             if [ -f ${D}${datadir}/installed-tests/glib/gdbus-serialization.test
>> ]; then
>>                     if ${@bb.utils.contains("DISTRO_FEATURES", "x11",
>> "false", "true", d)}; then
>>                             rm ${D}${datadir}/installed-tests
>> /glib/gdbus-serialization.test
>>                     fi
>>             fi
>>
>>       Obviously this isn't important for a native package.  Any
>>       suggestions on how I might keep this from creeping in?
>>
>>
>> The tests are only installed for target builds anyway, so that do_install
>> part could be target-specific.
>>
>
> How would one change the recipe to reflect that?  Is there an
> override that effectively says "not -native"?
>

do_install_append_class-target () should work.

 - Jussi

[-- Attachment #2: Type: text/html, Size: 2077 bytes --]

Re: sstate-cache and native recipes

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

On 18 January 2017 at 09:51, Gary Thomas <gary@mlbassoc.com> wrote:

> How would one change the recipe to reflect that?  Is there an
> override that effectively says "not -native"?
>

do_install_class-target.

Ross

[-- Attachment #2: Type: text/html, Size: 640 bytes --]

Re: sstate-cache and native recipes

[Kristian Amlie]

On 18/01/17 12:28, Burton, Ross wrote:
> 
> On 18 January 2017 at 09:51, Gary Thomas <gary@mlbassoc.com
> <mailto:gary@mlbassoc.com>> wrote:
> 
>     How would one change the recipe to reflect that?  Is there an
>     override that effectively says "not -native"?
> 
> 
> do_install_class-target.

This is exactly the annoyance I've hit as well. Did you have any success
with this?

I can probably give it a spin as well if you want me to.

-- 
Kristian

Re: sstate-cache and native recipes

[Gary Thomas]

On 2017-01-20 14:05, Kristian Amlie wrote:
> On 18/01/17 12:28, Burton, Ross wrote:
>>
>> On 18 January 2017 at 09:51, Gary Thomas <gary@mlbassoc.com
>> <mailto:gary@mlbassoc.com>> wrote:
>>
>>     How would one change the recipe to reflect that?  Is there an
>>     override that effectively says "not -native"?
>>
>>
>> do_install_class-target.
>
> This is exactly the annoyance I've hit as well. Did you have any success
> with this?
>
> I can probably give it a spin as well if you want me to.
>

I have a patch that I'll post tomorrow

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------