Splitting meta-oe

Splitting meta-oe

[Paul Eggleton]

Hi all,

The meta-oe layer [1] is becoming the home for additional recipes that we miss 
from OE-Classic that don't have another obvious layer to go into, and this is 
a good thing. However I think that meta-oe is already in a state where quite a 
few people feel the new structure is not working for them.

As I see it, meta-oe is trying provide the following:

 1) Additional recipes that are not in OE-Core

 2) Additional functionality for OE-Core recipes that we can't enable in OE-
Core itself (e.g. enabling postgres support in Qt) - mostly just a side-effect 
of #1

 3) A place to preserve old versions of recipes that have been removed from 
OE-Core in favour of newer versions

 4) Newer less-well tested versions of recipes

I think what most people want when they enable meta-oe in their layer 
configuration is #1, and it's probably OK to get #2 along with it. They do not 
however expect versions of toolchains, eglibc or other fairly fundamental bits 
and pieces that might cause their build to fail when everything worked fine 
just building with OE-Core (#4). Equally I expect there will be some people 
who want just #3 and nothing else.

I understand there is significant value in providing all of these things, I 
just think we shouldn't be trying to do them all in one largely indivisible 
layer. In our new layer structure we should be able to find a way to split the 
metadata so that people who want any combination can still have what we have 
in meta-oe today, and yet those who just want one of them don't get unexpected 
build failures or older/newer versions being built.

Thoughts?

Cheers,
Paul

[1] http://cgit.openembedded.org/meta-openembedded/tree/meta-oe

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Splitting meta-oe

[Frans Meulenbroeks]

2012/1/25 Paul Eggleton <paul.eggleton@linux.intel.com>

> Hi all,
>
> The meta-oe layer [1] is becoming the home for additional recipes that we
> miss
> from OE-Classic that don't have another obvious layer to go into, and this
> is
> a good thing. However I think that meta-oe is already in a state where
> quite a
> few people feel the new structure is not working for them.
>
> As I see it, meta-oe is trying provide the following:
>
>  1) Additional recipes that are not in OE-Core
>
>  2) Additional functionality for OE-Core recipes that we can't enable in
> OE-
> Core itself (e.g. enabling postgres support in Qt) - mostly just a
> side-effect
> of #1
>
>  3) A place to preserve old versions of recipes that have been removed from
> OE-Core in favour of newer versions
>
>  4) Newer less-well tested versions of recipes
>
> I think what most people want when they enable meta-oe in their layer
> configuration is #1, and it's probably OK to get #2 along with it. They do
> not
> however expect versions of toolchains, eglibc or other fairly fundamental
> bits
> and pieces that might cause their build to fail when everything worked fine
> just building with OE-Core (#4). Equally I expect there will be some people
> who want just #3 and nothing else.
>
> I understand there is significant value in providing all of these things, I
> just think we shouldn't be trying to do them all in one largely indivisible
> layer. In our new layer structure we should be able to find a way to split
> the
> metadata so that people who want any combination can still have what we
> have
> in meta-oe today, and yet those who just want one of them don't get
> unexpected
> build failures or older/newer versions being built.
>
> Thoughts?
>
> Cheers,
> Paul
>
> [1] http://cgit.openembedded.org/meta-openembedded/tree/meta-oe
>
> --
>
> Paul, all,

I feel this is a good proposal.
We might also attempt to define some criteria on what should go in and what
not.

Some more finegrained comments:

#3 should probably not be needed (it might be as a temporary transient).
I would hope/expect that meta-oe builds with oe-core and what is in there.
If a distro needs an older version of something (for whatever reason) it
better should be in the distro overlay.
Ditto if a board needs an older version.
(btw and if it is for historical reasons: the old versions can always be
retrieved from git).
(Note that I see no problem if there is a drastic change that needs some
work in meta-oe where an old version is kept as a transient to keep things
working while working to move to the new version)

Wrt #4: that is indeed something that people do not always expect. I feel a
better place for these newer/less-stable recipes would be a oe-core-next
branch or tree (assuming we are talking about oe-core recipes)

Best regards, Frans

Re: Splitting meta-oe

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2598 bytes --]

On Wed, Jan 25, 2012 at 12:48:11PM +0000, Paul Eggleton wrote:
> Hi all,
> 
> The meta-oe layer [1] is becoming the home for additional recipes that we miss 
> from OE-Classic that don't have another obvious layer to go into, and this is 
> a good thing. However I think that meta-oe is already in a state where quite a 
> few people feel the new structure is not working for them.
> 
> As I see it, meta-oe is trying provide the following:
> 
>  1) Additional recipes that are not in OE-Core
> 
>  2) Additional functionality for OE-Core recipes that we can't enable in OE-
> Core itself (e.g. enabling postgres support in Qt) - mostly just a side-effect 
> of #1
> 
>  3) A place to preserve old versions of recipes that have been removed from 
> OE-Core in favour of newer versions
> 
>  4) Newer less-well tested versions of recipes
> 
> I think what most people want when they enable meta-oe in their layer 
> configuration is #1, and it's probably OK to get #2 along with it. They do not 
> however expect versions of toolchains, eglibc or other fairly fundamental bits 
> and pieces that might cause their build to fail when everything worked fine 
> just building with OE-Core (#4). Equally I expect there will be some people 
> who want just #3 and nothing else.
> 
> I understand there is significant value in providing all of these things, I 
> just think we shouldn't be trying to do them all in one largely indivisible 
> layer. In our new layer structure we should be able to find a way to split the 
> metadata so that people who want any combination can still have what we have 
> in meta-oe today, and yet those who just want one of them don't get unexpected 
> build failures or older/newer versions being built.
> 
> Thoughts?

I don't see problems as it's now and I fear that splitting it even more
will result only in more complexity.

If there are problems with some recipes then we should try to fix them
no matter in which even smaller layer they belong.

I think we should rather address that issue with layer "priority" which
should be easier to understand and adjust in bblayers.conf.

And that DEFAULT_PREFERRENCE should be respected across layers, IIRC if
you add newer less-well tested version of some recipe (like I did with
libsoup-2.4 in meta-efl) then it's preferred over "stable" version in
oe-core even when it has lower D_P -1.

And we should continue to replace .bb files which exists in meta-oe and
also in oe-core with .bbappends.

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Splitting meta-oe

[Philip Balister]

On 01/25/2012 07:48 AM, Paul Eggleton wrote:
> 
> I think what most people want when they enable meta-oe in their layer 
> configuration is #1, and it's probably OK to get #2 along with it. They do not 
> however expect versions of toolchains, eglibc or other fairly fundamental bits 
> and pieces that might cause their build to fail when everything worked fine 
> just building with OE-Core (#4). Equally I expect there will be some people 
> who want just #3 and nothing else.

My understanding is the OE-Core toolchains lack the Linaro patches which
a extremely useful for people using armv7. So dropping toolchains from
meta-oe would be a really bad thing for a large portion of the user base.

Philip

Re: Splitting meta-oe

[Paul Eggleton]

On Wednesday 25 January 2012 09:34:53 Philip Balister wrote:
> On 01/25/2012 07:48 AM, Paul Eggleton wrote:
> > I think what most people want when they enable meta-oe in their layer
> > configuration is #1, and it's probably OK to get #2 along with it. They do
> > not however expect versions of toolchains, eglibc or other fairly
> > fundamental bits and pieces that might cause their build to fail when
> > everything worked fine just building with OE-Core (#4). Equally I expect
> > there will be some people who want just #3 and nothing else.
> 
> My understanding is the OE-Core toolchains lack the Linaro patches which
> a extremely useful for people using armv7. So dropping toolchains from
> meta-oe would be a really bad thing for a large portion of the user base.

I was suggesting moving these toolchains to a separate layer where those who 
don't need them aren't tripping over them. Adding such a layer to your 
configuration is more or less trivial, and once done you would not notice any 
difference.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Splitting meta-oe

[Paul Eggleton]

On Wednesday 25 January 2012 15:10:18 Martin Jansa wrote:
> I don't see problems as it's now and I fear that splitting it even more
> will result only in more complexity.

We deal with a high-level of complexity in OE, I don't think there's any 
getting away from it; and I'm not entirely convinced this is a significant 
increase. Debugging a build failure can be pretty complex - it can be a 
complete showstopper for someone just getting started, so let's try to avoid 
at least one class of failure that we can fairly easily avoid.

> If there are problems with some recipes then we should try to fix them
> no matter in which even smaller layer they belong.

Sure, this always applies. I just think it would be a worthwhile effort to 
reduce the likelihood of people coming across unexpected failures in parts of 
the system they don't need or would rather not have to debug themselves.
 
> I think we should rather address that issue with layer "priority" which
> should be easier to understand and adjust in bblayers.conf.

This is fine for advanced users like you and me who would know that adding a 
layer has introduced something they don't need and they should just adjust the 
priority to fix it. For the uninitiated, priority values in bblayers.conf will 
be yet more knobs they have to adjust before their build will work.

> And that DEFAULT_PREFERRENCE should be respected across layers, IIRC if
> you add newer less-well tested version of some recipe (like I did with
> libsoup-2.4 in meta-efl) then it's preferred over "stable" version in
> oe-core even when it has lower D_P -1.

I agree that in its most common usage DEFAULT_PREFERENCE is likely to be more 
valuable independent of the layer priority - we should consider this. The 
usefulness in this context is of course dependent on people actually setting 
DEFAULT_PREFERENCE = "-1" when they add the new recipe however.
 
> And we should continue to replace .bb files which exists in meta-oe and
> also in oe-core with .bbappends.

Agreed.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Splitting meta-oe

[Philip Balister]

On 01/25/2012 09:47 AM, Paul Eggleton wrote:
> On Wednesday 25 January 2012 09:34:53 Philip Balister wrote:
>> On 01/25/2012 07:48 AM, Paul Eggleton wrote:
>>> I think what most people want when they enable meta-oe in their layer
>>> configuration is #1, and it's probably OK to get #2 along with it. They do
>>> not however expect versions of toolchains, eglibc or other fairly
>>> fundamental bits and pieces that might cause their build to fail when
>>> everything worked fine just building with OE-Core (#4). Equally I expect
>>> there will be some people who want just #3 and nothing else.
>>
>> My understanding is the OE-Core toolchains lack the Linaro patches which
>> a extremely useful for people using armv7. So dropping toolchains from
>> meta-oe would be a really bad thing for a large portion of the user base.
> 
> I was suggesting moving these toolchains to a separate layer where those who 
> don't need them aren't tripping over them. Adding such a layer to your 
> configuration is more or less trivial, and once done you would not notice any 
> difference.

We should try and have a face to face meeting with OE developers at ELC
in a few weeks to talk about some of these issues.

Philip

Re: Splitting meta-oe

[Enrico]

On Wed, Jan 25, 2012 at 4:09 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Wednesday 25 January 2012 15:10:18 Martin Jansa wrote:
>> And that DEFAULT_PREFERRENCE should be respected across layers, IIRC if
>> you add newer less-well tested version of some recipe (like I did with
>> libsoup-2.4 in meta-efl) then it's preferred over "stable" version in
>> oe-core even when it has lower D_P -1.
>
> I agree that in its most common usage DEFAULT_PREFERENCE is likely to be more
> valuable independent of the layer priority - we should consider this. The
> usefulness in this context is of course dependent on people actually setting
> DEFAULT_PREFERENCE = "-1" when they add the new recipe however.

This would be very very useful.

Enrico

Re: Splitting meta-oe

[Frans Meulenbroeks]

I like the idea of putting toolchains in a layer.

Wrt DP = -1:
I feel if a recipe needs DP = -1 it is not good enough to be added to
meta-oe. Probably it is more something for an unstable layer.
(and DP = -1 encourages the growth of deadwood; I still recall in OE that
there were some recipes that had the last 3 or 4 versions all with DP = -1;
let's try to avoid this in oe-core/meta-oe)

Best regards, Frans.

Re: Splitting meta-oe

[Paul Eggleton]

On Wednesday 25 January 2012 11:00:53 Philip Balister wrote:
> We should try and have a face to face meeting with OE developers at ELC
> in a few weeks to talk about some of these issues.

Unfortunately I won't be at ELC in person. Time difference and technology 
allowing I could be there virtually or have someone who is there represent me, 
but I do think that we should be able to resolve most things here on the 
mailing lists (if not fully then at least as preparation for an in-person 
discussion).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Splitting meta-oe

[Khem Raj]

On Wed, Jan 25, 2012 at 4:48 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> As I see it, meta-oe is trying provide the following:
>
>  1) Additional recipes that are not in OE-Core
>
>  2) Additional functionality for OE-Core recipes that we can't enable in OE-
> Core itself (e.g. enabling postgres support in Qt) - mostly just a side-effect
> of #1
>
>  3) A place to preserve old versions of recipes that have been removed from
> OE-Core in favour of newer versions
>
>  4) Newer less-well tested versions of recipes

meta-oe was started to fill in gaps that oe-core would have for
existing oe users
since oe-core will not have all required functionality and so distros
could migrate
to using oe-core and not loose the bells and whistles they already have.
Plus its an umbrella layer which has many layers underneath.

However the problems it sees right now are that there are pieces which
could be considered
distro specific and ones, the new untested recipes should not be an
issue if they are pinned
properly. recipes that move our of oe-core I agree should be hosted in
a meta-deprecated or somesuch

I am happy to move out toolchain bits as well into a separate layer
under meta-openembedded umbrella
that will clarify it a bit and should not be a big hassle.

I think real issue people suffer from is 2 and the duplicate recipes
in meta-oe which has different configuration
in oe-core.

So meta-oe meta-toolchain meta-deprecated could be created now we need
to find maintainer for these layers.


All said if it was possible to handpick versions from layers and use
bitbake -g output pn-depends.dot
to generate some manifest so indicate which recipe is being picked
from a set of recipes from different layers would be helpful.
since there always will be some duplicate recipes no matter what.

Re: Splitting meta-oe

[Khem Raj]

On Wed, Jan 25, 2012 at 9:33 AM, Frans Meulenbroeks
<fransmeulenbroeks@gmail.com> wrote:
>
> Wrt DP = -1:
> I feel if a recipe needs DP = -1 it is not good enough to be added to
> meta-oe. Probably it is more something for an unstable layer.
> (and DP = -1 encourages the growth of deadwood; I still recall in OE that
> there were some recipes that had the last 3 or 4 versions all with DP = -1;
> let's try to avoid this in oe-core/meta-oe)

yes no DP = -1 in published layers thats almost useless.

Re: Splitting meta-oe

[Phil Blundell]

On Wed, 2012-01-25 at 17:41 +0000, Paul Eggleton wrote:
> On Wednesday 25 January 2012 11:00:53 Philip Balister wrote:
> > We should try and have a face to face meeting with OE developers at ELC
> > in a few weeks to talk about some of these issues.
> 
> Unfortunately I won't be at ELC in person. Time difference and technology 
> allowing I could be there virtually or have someone who is there represent me, 
> but I do think that we should be able to resolve most things here on the 
> mailing lists (if not fully then at least as preparation for an in-person 
> discussion).

Agreed.  I think the mailing lists ought to remain the primary forum for
decision-making.  If there's something that needs to be discussed, let's
do it here.  If we can't reach a consensus, that's what we have the TSC
for.

p.

Re: Splitting meta-oe

[Paul Eggleton]

On Wednesday 25 January 2012 09:49:14 Khem Raj wrote:
> meta-oe was started to fill in gaps that oe-core would have for
> existing oe users  since oe-core will not have all required functionality
> and so distros could migrate to using oe-core and not loose the bells and
> whistles they already have.

Sure, that's totally reasonable. I think we've built up some experience in the 
time we've had so far that we can apply in order to improve the structure a 
little though.

> Plus its an umbrella layer which has many layers underneath.

To save confusion, I speak only of the meta-oe layer within the meta-
openembedded repository. The meta-openembedded repository itself is not a 
layer of any kind, only the directories within it are. (I really wish we had 
chosen an entirely distinct name for the repository, but I guess that ship has 
sailed.)

> However the problems it sees right now are that there are pieces which
> could be considered distro specific and ones, the new untested recipes should
> not be an issue if they are pinned properly. 

So there ought not to be too much distro-specific in meta-oe; obviously there 
is some at the moment due to the small number of distros actively using it.

Unfortunately this pinning (I assume you mean PREFERRED_VERSION_) can only 
really occur within a distro that already includes meta-oe, and then you need 
to buy into the rest of the policy that that distro provides. If you are 
starting only from OE-Core or your distro does not typically include meta-oe, 
it will have no such pinning.

> recipes that move our of oe-core I agree should be hosted in
> a meta-deprecated or somesuch
>
> I am happy to move out toolchain bits as well into a separate layer
> under meta-openembedded umbrella
> that will clarify it a bit and should not be a big hassle.

Sounds great!

> I think real issue people suffer from is 2 and the duplicate recipes
> in meta-oe which has different configuration
> in oe-core.

I agree these do need to be well-justified, and should avoid being just 
manifestations of distro policy. These are gradually being ironed out where 
appropriate, we just need to continue with this process.

> So meta-oe meta-toolchain meta-deprecated could be created now we need
> to find maintainer for these layers.

It might be a bit presumptuous but I would nominate you for the toolchain 
layer ;) since you more or less maintain the recipes that would go in it 
already. (I guess meta-toolchain probably isn't the best name since that 
already means something in OE.)

As for the other layers I would be happy to assist Koen in maintaining them if 
he would like, but I think he's been doing a fine job with meta-oe already.

> All said if it was possible to handpick versions from layers and use
> bitbake -g output pn-depends.dot
> to generate some manifest so indicate which recipe is being picked
> from a set of recipes from different layers would be helpful.
> since there always will be some duplicate recipes no matter what.

The bitbake-layers tool is supposed to help at least with the reporting side 
of this, and it does report where bbappends exist and recipes have been 
overlayed. Right now however it does not report anything if a recipe is 
overlayed but the version number is different; I'm looking into fixing that at 
the moment. BTW I'd appreciate feedback on bitbake-layers as I've not had a 
lot since it was extended - feature requests welcome.

As an aside, we also still have to consider Richard's proposal for version-
independent bbappends (i.e. using % as a wildcard in the file name). I don't 
think anyone responded to that yet.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Splitting meta-oe

[Phil Blundell]

On Wed, 2012-01-25 at 15:45 -0500, Philip Balister wrote:
> On 01/25/2012 12:57 PM, Phil Blundell wrote:
> > Agreed.  I think the mailing lists ought to remain the primary forum for
> > decision-making.  If there's something that needs to be discussed, let's
> > do it here.  If we can't reach a consensus, that's what we have the TSC
> > for.
> 
> I do not want this to discourage from OE developers talking about things
> like this when they have the chance. Sometimes, high bandwidth face to
> face conversations can resolve issues much faster.

Absolutely, folks should feel welcome to discuss this and any other
issues whenever they have the chance.  And if no conclusion has emerged
by the time ELC comes around then that might indeed be a useful way to
move things forward.  But, such face-to-face meetings inevitably involve
only a subset of developers and the only way to ensure that all
interested parties are able to participate is to hold discussion on the
mailing list whenever possible.

p.

Re: Splitting meta-oe

[Philip Balister]

On 01/25/2012 04:03 PM, Phil Blundell wrote:
> On Wed, 2012-01-25 at 15:45 -0500, Philip Balister wrote:
>> On 01/25/2012 12:57 PM, Phil Blundell wrote:
>>> Agreed.  I think the mailing lists ought to remain the primary forum for
>>> decision-making.  If there's something that needs to be discussed, let's
>>> do it here.  If we can't reach a consensus, that's what we have the TSC
>>> for.
>>
>> I do not want this to discourage from OE developers talking about things
>> like this when they have the chance. Sometimes, high bandwidth face to
>> face conversations can resolve issues much faster.
> 
> Absolutely, folks should feel welcome to discuss this and any other
> issues whenever they have the chance.  And if no conclusion has emerged
> by the time ELC comes around then that might indeed be a useful way to
> move things forward.  But, such face-to-face meetings inevitably involve
> only a subset of developers and the only way to ensure that all
> interested parties are able to participate is to hold discussion on the
> mailing list whenever possible.

Sure. I don't think anyone is suggesting that a few guys get together
and make binding decisions, just try to work through things a little faster.

Philip

Re: Splitting meta-oe

[Khem Raj]

On Wed, Jan 25, 2012 at 10:39 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Wednesday 25 January 2012 09:49:14 Khem Raj wrote:
>> meta-oe was started to fill in gaps that oe-core would have for
>> existing oe users  since oe-core will not have all required functionality
>> and so distros could migrate to using oe-core and not loose the bells and
>> whistles they already have.
>
> Sure, that's totally reasonable. I think we've built up some experience in the
> time we've had so far that we can apply in order to improve the structure a
> little though.
>
>> Plus its an umbrella layer which has many layers underneath.
>
> To save confusion, I speak only of the meta-oe layer within the meta-
> openembedded repository. The meta-openembedded repository itself is not a
> layer of any kind, only the directories within it are. (I really wish we had
> chosen an entirely distinct name for the repository, but I guess that ship has
> sailed.)
>
>> However the problems it sees right now are that there are pieces which
>> could be considered distro specific and ones, the new untested recipes should
>> not be an issue if they are pinned properly.
>
> So there ought not to be too much distro-specific in meta-oe; obviously there
> is some at the moment due to the small number of distros actively using it.
>
> Unfortunately this pinning (I assume you mean PREFERRED_VERSION_) can only
> really occur within a distro that already includes meta-oe, and then you need
> to buy into the rest of the policy that that distro provides. If you are
> starting only from OE-Core or your distro does not typically include meta-oe,
> it will have no such pinning.
>

oe-core does have policies see default* files under meta/conf
however anyone who is using oe has to start with some policies iow distro
if its a custom one or if its predefined one

>> recipes that move our of oe-core I agree should be hosted in
>> a meta-deprecated or somesuch
>>
>> I am happy to move out toolchain bits as well into a separate layer
>> under meta-openembedded umbrella
>> that will clarify it a bit and should not be a big hassle.
>
> Sounds great!

toolchain is nothing but an example of conflicting recipes here.

>
>> I think real issue people suffer from is 2 and the duplicate recipes
>> in meta-oe which has different configuration
>> in oe-core.
>
> I agree these do need to be well-justified, and should avoid being just
> manifestations of distro policy. These are gradually being ironed out where
> appropriate, we just need to continue with this process.

yes this area needs some attention and I would request everyone to
clean this bit.

>
>> So meta-oe meta-toolchain meta-deprecated could be created now we need
>> to find maintainer for these layers.
>
> It might be a bit presumptuous but I would nominate you for the toolchain
> layer ;) since you more or less maintain the recipes that would go in it
> already. (I guess meta-toolchain probably isn't the best name since that
> already means something in OE.)
>

I think right now eglibc 2.12 is one of deprecated recipes that moved
into meta-oe
and I know angstrom currently depends on it. Sometimes if there are no more than
one layer who depends on a deprecared recipe it might be better to
move it to that layer itself.
so some careful decision would be involved when moving them around.
meta-deprecated may sound
like deprecated but it should actually be a maintained layer.

> As for the other layers I would be happy to assist Koen in maintaining them if
> he would like, but I think he's been doing a fine job with meta-oe already.
>
>> All said if it was possible to handpick versions from layers and use
>> bitbake -g output pn-depends.dot
>> to generate some manifest so indicate which recipe is being picked
>> from a set of recipes from different layers would be helpful.
>> since there always will be some duplicate recipes no matter what.
>
> The bitbake-layers tool is supposed to help at least with the reporting side
> of this, and it does report where bbappends exist and recipes have been
> overlayed. Right now however it does not report anything if a recipe is
> overlayed but the version number is different; I'm looking into fixing that at
> the moment. BTW I'd appreciate feedback on bitbake-layers as I've not had a
> lot since it was extended - feature requests welcome.
>

hmm I have not used it but I will try. Its I think very important that
user knows
which recipe it feeding into the build. Especially when there are
multiple recipes
in the layers he/she is using. Second thing would be .bbappends as to
how the recipe
was bbappended.

> As an aside, we also still have to consider Richard's proposal for version-
> independent bbappends (i.e. using % as a wildcard in the file name). I don't
> think anyone responded to that yet.
>

We should discuss that in separate thread.

> Cheers,
> Paul
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre

Re: Splitting meta-oe

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

On Wed, Jan 25, 2012 at 06:33:13PM +0100, Frans Meulenbroeks wrote:
> I like the idea of putting toolchains in a layer.
> 
> Wrt DP = -1:
> I feel if a recipe needs DP = -1 it is not good enough to be added to
> meta-oe. Probably it is more something for an unstable layer.
> (and DP = -1 encourages the growth of deadwood; I still recall in OE that
> there were some recipes that had the last 3 or 4 versions all with DP = -1;
> let's try to avoid this in oe-core/meta-oe)

I'm not proposing to stash multiple versions of same recipe with D_P in
same layer.

What I'm missing is use-case when for example with libsoup-2.4 I'm
adding unstable version, because actually webkit-efl has random runtime
crashes with stable version, but works better with unstable one.

So I want to be able to add recipe with D_P = -1 which would say, that
this recipe is not used unless someone really wants and adds P_V to his
layer to get it (I guess people using browsers based on webkit-efl).

But right now it will pick this unstable version from meta-oe by default
and you have to add P_V for that stable version in oe-core (and keep it
up2date as long as stable version in oe-core is updated).

And I guess the same use-case D_P should be used for all "not so well
tested" or "deprecated" recipes in meta-oe or any other layer.

Even if there is meta-deprecated layer.. I guess nobody would prefer all
deprecated recipes, but just some of them (ideally by pinning them with
P_V).

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Splitting meta-oe

[Frans Meulenbroeks]

2012/1/26 Martin Jansa <martin.jansa@gmail.com>

> On Wed, Jan 25, 2012 at 06:33:13PM +0100, Frans Meulenbroeks wrote:
> > I like the idea of putting toolchains in a layer.
> >
> > Wrt DP = -1:
> > I feel if a recipe needs DP = -1 it is not good enough to be added to
> > meta-oe. Probably it is more something for an unstable layer.
> > (and DP = -1 encourages the growth of deadwood; I still recall in OE that
> > there were some recipes that had the last 3 or 4 versions all with DP =
> -1;
> > let's try to avoid this in oe-core/meta-oe)
>
> I'm not proposing to stash multiple versions of same recipe with D_P in
> same layer.
>
> What I'm missing is use-case when for example with libsoup-2.4 I'm
> adding unstable version, because actually webkit-efl has random runtime
> crashes with stable version, but works better with unstable one.
>
> So I want to be able to add recipe with D_P = -1 which would say, that
> this recipe is not used unless someone really wants and adds P_V to his
> layer to get it (I guess people using browsers based on webkit-efl).
>
> But right now it will pick this unstable version from meta-oe by default
> and you have to add P_V for that stable version in oe-core (and keep it
> up2date as long as stable version in oe-core is updated).
>
> And I guess the same use-case D_P should be used for all "not so well
> tested" or "deprecated" recipes in meta-oe or any other layer.
>
> Even if there is meta-deprecated layer.. I guess nobody would prefer all
> deprecated recipes, but just some of them (ideally by pinning them with
> P_V).
>
>
> HI Martin,

I understand your reasoning.
My main concerns are:
- you might end up with lots of versions and no one knows that is actually
used (so it is difficult to clean it). We saw this in oe classic
- generally the reasons why a recipe has DP -1 are unclear. How would Joe
Average know that if he is using webkit-efl he is better off using this new
libsoup recipe?
- and if a recipe is not well enough tested it should probably not end up
in meta-oe anyway (but better e.g. in meta-oe-next or meta-oe-new or
whatever you want to call the super bleeding edge version)

Best regards, Frans.

Re: Splitting meta-oe

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2846 bytes --]

On Thu, Jan 26, 2012 at 03:45:52PM +0100, Frans Meulenbroeks wrote:
> 2012/1/26 Martin Jansa <martin.jansa@gmail.com>
> 
> > On Wed, Jan 25, 2012 at 06:33:13PM +0100, Frans Meulenbroeks wrote:
> > > I like the idea of putting toolchains in a layer.
> > >
> > > Wrt DP = -1:
> > > I feel if a recipe needs DP = -1 it is not good enough to be added to
> > > meta-oe. Probably it is more something for an unstable layer.
> > > (and DP = -1 encourages the growth of deadwood; I still recall in OE that
> > > there were some recipes that had the last 3 or 4 versions all with DP =
> > -1;
> > > let's try to avoid this in oe-core/meta-oe)
> >
> > I'm not proposing to stash multiple versions of same recipe with D_P in
> > same layer.
> >
> > What I'm missing is use-case when for example with libsoup-2.4 I'm
> > adding unstable version, because actually webkit-efl has random runtime
> > crashes with stable version, but works better with unstable one.
> >
> > So I want to be able to add recipe with D_P = -1 which would say, that
> > this recipe is not used unless someone really wants and adds P_V to his
> > layer to get it (I guess people using browsers based on webkit-efl).
> >
> > But right now it will pick this unstable version from meta-oe by default
> > and you have to add P_V for that stable version in oe-core (and keep it
> > up2date as long as stable version in oe-core is updated).
> >
> > And I guess the same use-case D_P should be used for all "not so well
> > tested" or "deprecated" recipes in meta-oe or any other layer.
> >
> > Even if there is meta-deprecated layer.. I guess nobody would prefer all
> > deprecated recipes, but just some of them (ideally by pinning them with
> > P_V).
> >
> >
> > HI Martin,
> 
> I understand your reasoning.
> My main concerns are:
> - you might end up with lots of versions and no one knows that is actually
> used (so it is difficult to clean it). We saw this in oe classic

I don't see this happening in oe-core or meta-oe, your grep proves it.

> - generally the reasons why a recipe has DP -1 are unclear. How would Joe
> Average know that if he is using webkit-efl he is better off using this new
> libsoup recipe?

Reading it about DP setting or in commit where I've added it?

> - and if a recipe is not well enough tested it should probably not end up
> in meta-oe anyway (but better e.g. in meta-oe-next or meta-oe-new or
> whatever you want to call the super bleeding edge version)

How will Joe Average know that because of webkit-efl he should enable
meta-efl-next (or
oe-core-next-libsoup-development-version-alpha-centauri-edition layer) 
 if he doesn't even know what's happening in layers he is
using already (meta-efl for webkit-efl)?

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Splitting meta-oe

[Andreas Oberritter]

On 26.01.2012 17:11, Martin Jansa wrote:
> On Thu, Jan 26, 2012 at 03:45:52PM +0100, Frans Meulenbroeks wrote:
>> - and if a recipe is not well enough tested it should probably not end up
>> in meta-oe anyway (but better e.g. in meta-oe-next or meta-oe-new or
>> whatever you want to call the super bleeding edge version)
> 
> How will Joe Average know that because of webkit-efl he should enable
> meta-efl-next (or
> oe-core-next-libsoup-development-version-alpha-centauri-edition layer) 
>  if he doesn't even know what's happening in layers he is
> using already (meta-efl for webkit-efl)?

I think libsoup is not a good example, because the specific version
doesn't live in meta-oe, but in meta-efl, which is fine IMO.

Being someone who's just in the process of moving a distribution from
OE-classic to OE-core + meta-oe, I've already been busy figuring out the
exact differences between recipes present in both layers. I'd welcome
any reduction in duplication and versions present in both layers.

Regards,
Andreas

Re: Splitting meta-oe

[Khem Raj]

On Thu, Jan 26, 2012 at 1:02 AM, Martin Jansa <martin.jansa@gmail.com> wrote:
>
> What I'm missing is use-case when for example with libsoup-2.4 I'm
> adding unstable version, because actually webkit-efl has random runtime
> crashes with stable version, but works better with unstable one.
>
> So I want to be able to add recipe with D_P = -1 which would say, that
> this recipe is not used unless someone really wants and adds P_V to his
> layer to get it (I guess people using browsers based on webkit-efl).
>
> But right now it will pick this unstable version from meta-oe by default
> and you have to add P_V for that stable version in oe-core (and keep it
> up2date as long as stable version in oe-core is updated).
>
> And I guess the same use-case D_P should be used for all "not so well
> tested" or "deprecated" recipes in meta-oe or any other layer.

yes thats a good thing to do. But I still would prefer a layer separation
from meta-oe
>
> Even if there is meta-deprecated layer.. I guess nobody would prefer all
> deprecated recipes, but just some of them (ideally by pinning them with
> P_V).
>

yes meta-deprecated by default should be added to BBMASK

> Cheers,
>
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: Splitting meta-oe

[Paul Eggleton]

On Wednesday 25 January 2012 12:48:11 Paul Eggleton wrote:
> The meta-oe layer [1] is becoming the home for additional recipes that we
> miss from OE-Classic that don't have another obvious layer to go into, and
> this is a good thing. However I think that meta-oe is already in a state
> where quite a few people feel the new structure is not working for them.
> 
> As I see it, meta-oe is trying provide the following:
> 
>  1) Additional recipes that are not in OE-Core
> 
>  2) Additional functionality for OE-Core recipes that we can't enable in OE-
> Core itself (e.g. enabling postgres support in Qt) - mostly just a
> side-effect of #1
> 
>  3) A place to preserve old versions of recipes that have been removed from
> OE-Core in favour of newer versions
> 
>  4) Newer less-well tested versions of recipes
> 
> I think what most people want when they enable meta-oe in their layer
> configuration is #1, and it's probably OK to get #2 along with it. They do
> not however expect versions of toolchains, eglibc or other fairly
> fundamental bits and pieces that might cause their build to fail when
> everything worked fine just building with OE-Core (#4). Equally I expect
> there will be some people who want just #3 and nothing else.
> 
> I understand there is significant value in providing all of these things, I
> just think we shouldn't be trying to do them all in one largely indivisible
> layer. In our new layer structure we should be able to find a way to split
> the metadata so that people who want any combination can still have what we
> have in meta-oe today, and yet those who just want one of them don't get
> unexpected build failures or older/newer versions being built.
> 
> Thoughts?
>
> [1] http://cgit.openembedded.org/meta-openembedded/tree/meta-oe

Koen, as the maintainer of meta-oe would you like to comment on this thread?

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Splitting meta-oe

[Koen Kooi]

Op 1 feb. 2012, om 11:07 heeft Paul Eggleton het volgende geschreven:

> On Wednesday 25 January 2012 12:48:11 Paul Eggleton wrote:
>> The meta-oe layer [1] is becoming the home for additional recipes that we
>> miss from OE-Classic that don't have another obvious layer to go into, and
>> this is a good thing. However I think that meta-oe is already in a state
>> where quite a few people feel the new structure is not working for them.
>> 
>> As I see it, meta-oe is trying provide the following:
>> 
>> 1) Additional recipes that are not in OE-Core
>> 
>> 2) Additional functionality for OE-Core recipes that we can't enable in OE-
>> Core itself (e.g. enabling postgres support in Qt) - mostly just a
>> side-effect of #1
>> 
>> 3) A place to preserve old versions of recipes that have been removed from
>> OE-Core in favour of newer versions
>> 
>> 4) Newer less-well tested versions of recipes
>> 
>> I think what most people want when they enable meta-oe in their layer
>> configuration is #1, and it's probably OK to get #2 along with it. They do
>> not however expect versions of toolchains, eglibc or other fairly
>> fundamental bits and pieces that might cause their build to fail when
>> everything worked fine just building with OE-Core (#4). Equally I expect
>> there will be some people who want just #3 and nothing else.
>> 
>> I understand there is significant value in providing all of these things, I
>> just think we shouldn't be trying to do them all in one largely indivisible
>> layer. In our new layer structure we should be able to find a way to split
>> the metadata so that people who want any combination can still have what we
>> have in meta-oe today, and yet those who just want one of them don't get
>> unexpected build failures or older/newer versions being built.
>> 
>> Thoughts?
>> 
>> [1] http://cgit.openembedded.org/meta-openembedded/tree/meta-oe
> 
> Koen, as the maintainer of meta-oe would you like to comment on this thread?

I'm waiting for FOSDEM and ELC to discuss this face to face.

Re: Splitting meta-oe

[Otavio Salvador]

On Wed, Feb 1, 2012 at 08:33, Koen Kooi <koen@dominion.thruhere.net> wrote:

> I'm waiting for FOSDEM and ELC to discuss this face to face.


I won't be on those events but I'd like to participate of the discussion so
please use the mailing list.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: Splitting meta-oe

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Op 01-02-12 12:11, Otavio Salvador schreef:
> On Wed, Feb 1, 2012 at 08:33, Koen Kooi <koen@dominion.thruhere.net>
> wrote:
> 
>> I'm waiting for FOSDEM and ELC to discuss this face to face.
> 
> 
> I won't be on those events but I'd like to participate of the discussion
> so please use the mailing list.

THat's fine, but I still won't be participating in this thread till after
the face to face meetings.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: GPGTools - http://gpgtools.org

iEYEARECAAYFAk8pXKoACgkQMkyGM64RGpGmoACfURPviYRIL0qWqgiMSXe2t3GF
mUkAoLCU+LASpI/JuHTjVR//2qpckxnx
=TZCM
-----END PGP SIGNATURE-----

Re: Splitting meta-oe

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Op 01-02-12 12:11, Otavio Salvador schreef:
> On Wed, Feb 1, 2012 at 08:33, Koen Kooi <koen@dominion.thruhere.net>
> wrote:
> 
>> I'm waiting for FOSDEM and ELC to discuss this face to face.
> 
> 
> I won't be on those events but I'd like to participate of the discussion
> so please use the mailing list.

Philip, Paul and I managed to squeeze in some time today to discuss all this
and the consensus matches Pauls proposal.

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: GPGTools - http://gpgtools.org

iEYEARECAAYFAk8umS8ACgkQMkyGM64RGpHfZQCfZ4r468JFyOgjYeO4lcvp+I9A
fpUAn3IK0r+IDujudIJmaNxqiaNxZ0ry
=d+TI
-----END PGP SIGNATURE-----