Is there a relationship between the sstate and the machine?

Is there a relationship between the sstate and the machine?

[Mans Zigher]

Hi,

I am using a build environment based on the yocto project from one of
the big HW suppliers in the mobile industries. They are continuously
breaking the principles behind the yocto project and at one point they
managed to break the sstate cache because they are doing things in
there own way instead of using what already exists. They managed to
fix the sstate cache but it now looks to only work when using there
machines when I am defining my own machine even though it is a copy of
one of theirs the sstate cache it is not working. The sstate cache is
detect and the SSTATE_DIR is correct but it finds only a handful of
matches and builds the rest from scratch. Is there a logical
explanation for this? Or is this yet again some custom crap that this
supplier have done. FYI if you know what supplier I am talking about I
would very much recommend staying the hell away from them this is the
worst linux distro I have ever worked with both on the build and on
the system it is some kind of freak combination of a normal linux
system and android worst combination ever.

Thanks

Re: [yocto] Is there a relationship between the sstate and the machine?

[Mikko Rapeli]

Hi,

On Wed, May 13, 2020 at 10:36:49AM +0200, Mans Zigher wrote:
> I am using a build environment based on the yocto project from one of
> the big HW suppliers in the mobile industries. They are continuously
> breaking the principles behind the yocto project and at one point they
> managed to break the sstate cache because they are doing things in
> there own way instead of using what already exists. They managed to
> fix the sstate cache but it now looks to only work when using there
> machines when I am defining my own machine even though it is a copy of
> one of theirs the sstate cache it is not working. The sstate cache is
> detect and the SSTATE_DIR is correct but it finds only a handful of
> matches and builds the rest from scratch. Is there a logical
> explanation for this? Or is this yet again some custom crap that this
> supplier have done. FYI if you know what supplier I am talking about I
> would very much recommend staying the hell away from them this is the
> worst linux distro I have ever worked with both on the build and on
> the system it is some kind of freak combination of a normal linux
> system and android worst combination ever.

This is a common problem. It is likely that I know which SoC supplier you
are talking about and have had the exact same issues. We have managed to
fix or workaround all of them. In most cases by reducing what the SoC layers
do by BBMASK'ing all non-essential BSP recipes, reverse engineering the
machine configuration and bbclasses to only use bare minimal ones.

I can only hope that our feedback in terms of patches and workarounds also
going back the supplier chain(s) and you at some point see the same patches.
For example we updated the yocto open source layers from 2.5 sumo to 3.0 zeus
with very minimal support from the SoC and other suppliers and their layers
are still only compatible with 2.5 sumo, officially. Also switched from
using a custom Android based binary toolchain(s there were multiple
at some point) to using meta-clang.

If you have detailed question or problems, it would be nice to discuss them
here, but I know that some details may be closed. At least some general
aspects how to debug issues may be discussed without exposing too much
details or, well, company names.

Cheers,

-Mikko

Re: [yocto] Is there a relationship between the sstate and the machine?

[Mans Zigher]

Thanks for your answer. Yes I think it is the same SoC supplier since
there is clang toolchain and they deliver it in a binary form and they
have supplied an class for using it. I will look into the meta-clang
that sounds like an interesting alternative. Regarding the BBMASK'ing
which I have used earlier but when trying to use it this time the
dependencies was really difficult to handle I finally managed to get
something working but in the end I ended up missing some packages
required to use there PC tools. I would definitely like to get some
more information about your experience and how you have handled some
parts of this build so I will open a new thread regarding dose
specific questions.

When trying to figure out how the sstate cache actually works the way
I understand it is that it is managed per task and not per recipe
which makes it even more unclear why I am not getting the expected
result. I am building once from scratch getting a sstate cache I then
remove the entire bitbake build directory and rebuild but I am reusing
the previous sstate cache and I am seeing a 99% match. Now if I take
the same cache and supply it to our CI I am getting a 16% match. Am I
missing something here? I am currently not using the SSTATE_MIRROR
instead I am currently just copying the cache over for testing. Is
this expected or is this also related to the this distro somehow?

Thanks,

Den ons 13 maj 2020 kl 10:59 skrev <Mikko.Rapeli@bmw.de>:
>
> Hi,
>
> On Wed, May 13, 2020 at 10:36:49AM +0200, Mans Zigher wrote:
> > I am using a build environment based on the yocto project from one of
> > the big HW suppliers in the mobile industries. They are continuously
> > breaking the principles behind the yocto project and at one point they
> > managed to break the sstate cache because they are doing things in
> > there own way instead of using what already exists. They managed to
> > fix the sstate cache but it now looks to only work when using there
> > machines when I am defining my own machine even though it is a copy of
> > one of theirs the sstate cache it is not working. The sstate cache is
> > detect and the SSTATE_DIR is correct but it finds only a handful of
> > matches and builds the rest from scratch. Is there a logical
> > explanation for this? Or is this yet again some custom crap that this
> > supplier have done. FYI if you know what supplier I am talking about I
> > would very much recommend staying the hell away from them this is the
> > worst linux distro I have ever worked with both on the build and on
> > the system it is some kind of freak combination of a normal linux
> > system and android worst combination ever.
>
> This is a common problem. It is likely that I know which SoC supplier you
> are talking about and have had the exact same issues. We have managed to
> fix or workaround all of them. In most cases by reducing what the SoC layers
> do by BBMASK'ing all non-essential BSP recipes, reverse engineering the
> machine configuration and bbclasses to only use bare minimal ones.
>
> I can only hope that our feedback in terms of patches and workarounds also
> going back the supplier chain(s) and you at some point see the same patches.
> For example we updated the yocto open source layers from 2.5 sumo to 3.0 zeus
> with very minimal support from the SoC and other suppliers and their layers
> are still only compatible with 2.5 sumo, officially. Also switched from
> using a custom Android based binary toolchain(s there were multiple
> at some point) to using meta-clang.
>
> If you have detailed question or problems, it would be nice to discuss them
> here, but I know that some details may be closed. At least some general
> aspects how to debug issues may be discussed without exposing too much
> details or, well, company names.
>
> Cheers,
>
> -Mikko