Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

[Jasper Orschulko <Jasper.Orschulko@iris-sensing.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Alex,

we are getting a bit off topic here, but whatever... ;) 

> What I mean is that I would try harder to find a workable setup while
> keeping the recipes for individual items that need to be built.

the individual recipes don't really add that much benefit for us,
mostly just maintenance overhead and complexity, as we need to
constantly ensure that the various components depend on each other in
the right order. The only beneficial aspect to the multitude of recipes
is the sstate cache. However, as the packages heavily depend on one
another, the benefit is marginal, as we have to rebuild most of the
application anyways.

Additionally, this is not how the developers build the application, so
basically we currently have to maintain two separate build workflows.
Changing to repo also improves the developer workflow outside of yocto,
e.g. thanks to the notion of "topics". The "devtool" option, whilst
duable I assume, would again limit the usability to within the Yocto
environment. We are currently trying to get to a point, where we manage
most of our (automated) workloads (e.g. testing) within Yocto but
historically this was not the case and for daily development this is
just not really practical.

> Particularly this:
> "if you want to create another version with different cmake flags,
> you would have to create copies of all 50ish recipes"
> looks odd. Why would you make copies? Just set the needed flags with
> a variable that is set from a global config,
> perhaps the distro or local.conf.

As far as I understand, this is not reasonably possible (though I might
be wrong. TBH, understanding variable scopes in Yocto is a nightmare).

To give this some context, our current base setup looks as follows:

- - We have one distro.conf
- - We have three image recipes with slightly different config +
featureset
- - AFAIK an image recipe cannot set variables within another recipe, it
just "consumes" existing recipes with their configuration.
- -> Thus we have three respective recipes with slightly different
configuration. If we where to use individual recipes in this setup,
we'd have 50ish x 3 recipes.

The approach you are describing might circumvent this particular
situation but basically just moves the issue around, as you're now
unabe to build all three configurations in one go, but need to start
juggling your local.confs. Alternatively you might think about using
multi-confs. However, each multiconf doubles the parse times for your
recipes. As we already use a couple of multiconfs for different machine
configs, we would basically increase the time it takes to parse the
recipes by ~fifteen-fold (5 pre-existing multi-confs x 3 (or more)
disto confs). Additionally, you now need to start worrying about
artifact names, as by default the distro name is not included.

So yeah... as far as I can tell, there are multiple ways to approach
this issue, but none of them seem straight forward nor pretty. Bitbake
as it is is just fundamentally not good at handling highly dynamic
configurations. The combination with KAS somewhat defuses the
situation, but there are still some situations where there is no easy
answer.

- -- 
With best regards

Jasper Orschulko
DevOps Engineer

Tel. +49 30 58 58 14 265
Fax +49 30 58 58 14 999
Jasper.Orschulko@iris-sensing.com

• • • • • • • • • • • • • • • • • • • • • • • • • •

iris-GmbH
infrared & intelligent sensors
Schnellerstraße 1-5 | 12439 Berlin

https://iris-sensing.com/





On Fri, 2021-11-05 at 19:45 +0100, Alexander Kanavin wrote:
> On Fri, 5 Nov 2021 at 19:05, Jasper Orschulko <jasper@fancydomain.eu>
> wrote:
> > 
> > But that is exactly what we are doing, by integrating the repo
> > fetcher into the yocto workflow, thus improving the yocto tooling
> > :)
> > Why reinvent the wheel, when you can reuse whats already there? You
> > wouldn't reinvent git just for yocto, would you?
> > 
> 
> 
> What I mean is that I would try harder to find a workable setup while
> keeping the recipes for individual items that need to be built. 
> For instance, devtool is capable of updating source revisions in
> recipes automatically, it may not work exactly as you want, but that
> can be fixed.
> Yes, repo itself is not proprietary, but your organizational workflow
> is.
> 
> Particularly this:
> "if you want to create another version with different cmake flags,
> you would have to create copies of all 50ish recipes"
> looks odd. Why would you make copies? Just set the needed flags with
> a variable that is set from a global config,
> perhaps the distro or local.conf.
> 
> Alex
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE4WyPMIC5Ap4+Ooo1Ygqew07VMNUFAmGFlPMACgkQYgqew07V
MNXoCAf/fN4gi1dfx7r4acfbu7kaw9+3rjdMu34J6SX/ahGjCfbTZAg1Twf8N6RZ
LBuOpDkXPU77iJuJixZySve5EZc4A2/NS0hqXEpD0aWf8AXaZzxq0UbEBM78oTgH
Hk1zVEFxwLwEHpVIQyvwx5rFz/JvZ2EN4aH576ciPaw5n+bHIZBevUfhssPrjQWX
yUTuWyeUoMJlXz41E+JfSd1VIQOC3hdWKF26er152zSyhYvhCagBMsHZMTKdrahn
2jmInGMSMizXR6vXBUTfwaB48Ba0iqFoi/tr7+jNIHV5WZNx4AvU61UOu8Lca6he
bGvM5nM7xRpeHuNX2xa9xWs3SkUNEg==
=UJaE
-----END PGP SIGNATURE-----