All of lore.kernel.org
 help / color / mirror / Atom feed
* Yocto Project and OE - Where now?
@ 2011-01-14 17:49 Richard Purdie
  2011-01-17 15:15 ` Koen Kooi
  2011-01-17 19:01 ` Frans Meulenbroeks
  0 siblings, 2 replies; 46+ messages in thread
From: Richard Purdie @ 2011-01-14 17:49 UTC (permalink / raw)
  To: openembedded-devel, openembedded-members

The vote result showed a strong opinion that Yocto and OE need to find a
way to work together which is something I'm very happy to see. I think
everyone is waiting for each other to make a move so I'm going to put
forward some ideas/discussion to try and start the ball rolling. I'm
filling out ideas here but I want to make it clear this is a straw-man
to discuss. I have given this a lot of thought though as its attention
to many of the details that will determine its success.

The agreement for OE to take up a seat on the Yocto Steering Group seems
to be progressing and I think that part of things is moving forward well
and the board is handling it. At a technical level we need to have some
discussion though.

I think the idea in people's minds is to create an "openembedded-core"
which would have a new development structure. There are various aims for
doing this but they include:

* Formalising processes for making changes
* Creation of a subset of metadata that has a consistent high quality 
  standard:
    - removal of legacy components (pkgconfig hacks, libtool hacks, 
      legacy staging, unneeded compiler arguments)
    - consistent metadata layout, spacing, use of variables
    - migration away from outdated practises (e.g. use BBCLASSEXTEND)
* Formalise cycles of development, stabilisation and release
* Ensure a form of standardised basic testing of changes and over time, 
  extend that testing
 
We've talked about doing this at OEDEMs in the past, we keep talking
around the issue, we now have an opportunity to attempt it and to make
it a success. Part of the problem was always manpower to undertake some
of it, Yocto gives us a mechanism to help with that.

What would we have to do to make that happen? Roughly speaking I think
the steps look like:

* Agreed to try a new contributions model
* Setup an openembedded-core repo
* Populate that repo with some base data
* Decide where OE core patches would be queued, discussed and processed
* Document the process
* Start using it
* Profit!

This raises some questions:

* What would the new contributions model be?
* Where should the oe-core repo be hosted?
* What do we populate the base repo with?
* Do we need a new mailing list to make it clear what core changes are 
  being proposed?
* Create a openembedded-core-contrib repo?

Assuming this happens there is then the question of what happens in
OpenEmbedded and what happens in Yocto to adapt to this.

I'd like to propose we take a significant chunk of Poky as the basis for
OE-core simply because we're already spent a significant amount of
effort cleaning it up and turning it into something that is close to
what we'd like to see from oe-core. There would be changes including:

* Strip out bitbake
* Strip out the poky glue components (scripts directory, documentation
directory, meta-emenlow, 
* Strip out the "poky" distro specific pieces
* Anything else we need to change to ensure the transition
* Add/remove recipes from the "core" as needed after discussion about 
  exactly what we need in there. I'm conscious for example 
  meta-toolchain-qte is missing and there is someone looking at fixing 
  that.
* Compare the existing OE classes with the ones in OE-core and ensure 
  nothing important is missing in OE-core.

I'd then propose the Poky repo continue to exist but effectively it
would become an automated amalgamation of bitbake + oe-core + poky/yocto
glue, hopefully just taking commits from the various upstream repos.
This would fulfil Yocto's requirement of making something simple to use
for end users but allow us to share much of the work between OE and
Yocto. Whether there are existing tools that would help with the
creation of such an automated repo or whether its a case of just writing
a script, I don't know but we'd work something out.

For OE, we've talked about cleaning it out for a long time and I think
meta-openembedded is the solution there with some cleanup happening as
people transition the recipes they use.

The one thing that is still bugging me a little is handling of layers
from a practical point of view. I like the idea of layer repositories
with a specific topic and defined ownership. On the other hand I want to
keep things simple for users both in checking out the build system and
for contributing changes back. There are a variety of solutions using
various git tools and I'd really like to have the discussion where we
compare the different options and figure out how to make a great user
experience.

I don't want to block making progress on the Yocto/OE relationship on
that one issue though and am happy that the poky specific repo problem
can be scripted for example in the short term until we arrive at the
full solution.

I'd also like to take a look at the practicalities of the contributions
model changes this would need. The free for all everyone can push model
of OE as it stands today is not really suited to achieve the aims
mentioned above. I believe we can look to other projects such as the
Linux kernel for examples of different models which may assist with
this. There, a "pull" model is used to great effect. In Yocto we already
have a tree like setup through which patches get submitted into Poky
with people performing review, then aggregating changes. I act as the
final point of that chain. I can't claim we have everything perfect but
I do think it works well. It means that we can do things like notice
instability in the codebase and throttle changes until those
instabilities are addressed. It wouldn't be too much change for me to
undertake that role with the OE-core instead of Yocto/Poky. I have the
appropriate understanding of the metadata core and my position as an LF
fellow places me somewhere independent with a clear mandate to spend
time on things like this. I also have the required passion for the OE
architecture and desire for OE to be successful!

One other question also springs to mind which is what is the TSC's remit
in this new contribution model. Its a complex issue, the TSC has helped
in cases but its also not without its problems, particularly its members
habit of not getting around to the "paperwork" of which I'm as guilty as
anyone else :(.

Also, there are now a number of people in the Yocto community who's
technical experience and background in embedded is already strong and OE
experience is growing ever stronger yet at present they're ineligible
for a seat on the TSC as they're not OE e.V. members or known much in
the OE community. The fact they're not e.V. members was a conscious
decision as an "invasion" of people from Yocto would send a message to
the OE community that simply wouldn't be correct. I mention this not as
any form of complaint but just to make the different parts of the issue
clear.

I'm open to ideas about how to move this forward and will also ask this
question of the TSC itself. Yocto's approach on this is quite different
with the steering group taking a hands off approach and the day to day
running being left to the architects and maintainers groups.

Finally, where should this discussion happen? The TSC likely feels some
of the issues discussed in this email are for them to reach a decision
on and provide advice to the wider OE community. I'd agree the TSC does
have remit over a lot of topics discussed here but equally I don't feel
it inappropriate to discuss them with the members list too.

I am conscious that openembedded-devel isn't likely to be seeing any
discussions on the members list and there may be viewpoints there. I'd
hate to think people felt ignored or under represented as that isn't
intentional and more an artefact of OE's structure. I'd appreciate some
guidance on where people think this discussion should happen. For now
I'd cc'ing both the members and devel list.

Ultimately, I think the only way we're going to move forward with this
is to actually try something.

Cheers,

Richard




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-14 17:49 Yocto Project and OE - Where now? Richard Purdie
@ 2011-01-17 15:15 ` Koen Kooi
  2011-01-17 19:01 ` Frans Meulenbroeks
  1 sibling, 0 replies; 46+ messages in thread
From: Koen Kooi @ 2011-01-17 15:15 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel


Op 14 jan 2011, om 18:49 heeft Richard Purdie het volgende geschreven:

> The vote result showed a strong opinion that Yocto and OE need to find a
> way to work together which is something I'm very happy to see. I think
> everyone is waiting for each other to make a move so I'm going to put
> forward some ideas/discussion to try and start the ball rolling. I'm
> filling out ideas here but I want to make it clear this is a straw-man
> to discuss. I have given this a lot of thought though as its attention
> to many of the details that will determine its success.
> 
> The agreement for OE to take up a seat on the Yocto Steering Group seems
> to be progressing and I think that part of things is moving forward well
> and the board is handling it. At a technical level we need to have some
> discussion though.
> 
> I think the idea in people's minds is to create an "openembedded-core"
> which would have a new development structure. There are various aims for
> doing this but they include:
> 
> * Formalising processes for making changes
> * Creation of a subset of metadata that has a consistent high quality 
>  standard:
>    - removal of legacy components (pkgconfig hacks, libtool hacks, 
>      legacy staging, unneeded compiler arguments)
>    - consistent metadata layout, spacing, use of variables
>    - migration away from outdated practises (e.g. use BBCLASSEXTEND)
> * Formalise cycles of development, stabilisation and release
> * Ensure a form of standardised basic testing of changes and over time, 
>  extend that testing
> 
> We've talked about doing this at OEDEMs in the past, we keep talking
> around the issue, we now have an opportunity to attempt it and to make
> it a success. Part of the problem was always manpower to undertake some
> of it, Yocto gives us a mechanism to help with that.
> 
> What would we have to do to make that happen? Roughly speaking I think
> the steps look like:
> 
> * Agreed to try a new contributions model
> * Setup an openembedded-core repo
> * Populate that repo with some base data
> * Decide where OE core patches would be queued, discussed and processed
> * Document the process
> * Start using it
> * Profit!

I think we can start with the following right now:

* setup an oe-core repo
* populate that repo

What I propose to do in parallel to that:

* setup oe-yocto-integration mailinglist
* people interesting in *working*[1] on the integration may subscribe
* hook that ml up to patchwork as a new project

That only leaves:

* contributions model discussion
* patch processing discussion
* documentation
* profit

> For OE, we've talked about cleaning it out for a long time and I think
> meta-openembedded is the solution there with some cleanup happening as
> people transition the recipes they use.

Apart from the weird pythondir() problem that came up last week, meta-openembedded is working quite well as layer on top of yocto. It would be nice to see more people contribution to it, though.

[snip]

> One other question also springs to mind which is what is the TSC's remit
> in this new contribution mode

As we discussed during OEDEM, the TSC needs to go through an election cycle to get some new people on. The current TSC is opaque, non-responsive and out of touch. I'm not sure a new election is going to help fix that, but it's worth a try.

> Ultimately, I think the only way we're going to move forward with this
> is to actually try something.

So lets get started!


[1] Lots of people have love to suggest colours for the bikeshed, but few people actually want to do actual painting


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-14 17:49 Yocto Project and OE - Where now? Richard Purdie
  2011-01-17 15:15 ` Koen Kooi
@ 2011-01-17 19:01 ` Frans Meulenbroeks
  2011-01-18  7:21   ` Graeme Gregory
  1 sibling, 1 reply; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-17 19:01 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel

Richard,

Thanks for your post.
I think you describe the issues at hand quite clearly.
I would definitely be interested in working towards a new structure.

The one thing that I found partially missing in your post are the
actual design guidelines.

You mention
> * Creation of a subset of metadata that has a consistent high quality
>  standard:
>    - removal of legacy components (pkgconfig hacks, libtool hacks,
>      legacy staging, unneeded compiler arguments)
>    - consistent metadata layout, spacing, use of variables
>    - migration away from outdated practises (e.g. use BBCLASSEXTEND)

I fully agree with these but there are several other things in OE
metadata that we should discuss and probably try to avoid.
Without the aim of trying to be complete, I would like to name some,
also with the aim to kick off the discussion on these, order to come
to an accepted set of guidelines.

- where possible stick to one recipe per package. This reduces the
maintenance work and reduces the QA nightmare of lots of different
permutations.
I feel one recipe per package should be the common case for
applications, and preferably also for libs (although I am well aware
that especially in the latter case multiple versions cannot always be
avoided).
For kernel, u-boot and some toolchain parts , multiple recipes are unavoidable.
When moving to a new version or when a recipe needs some more testing,
I can imagine two versions existing for a while, but I feel this
should be an exception.
If a new version is not stable enough to be in the mainline, it should
probably live in its own overlay, not in the main repos

We should try to avoid the version diarrhea that some packages
currently have in OE (e.g. I recall that at some point we had about 10
abiword recipes, I can not really imagine anyone knowing the
differences between these, and I fail to see the point in having
these). There is no need to keep old versions around, as they are
still retrievable from git (assuming git is used as SCM).

- avoid DEFAULT_PREFERENCE. For packages with only one recipe it
shouldn't be needed at all. New, not integrated or only partially
tested recipes should reside in a separate overlay. Machine
maintainers should know best what kernel and u-boot version works best
for their machine. We do probably need to come up with a model on how
to deal with version upgrades though. This could be done with
different layers (e.g. bsp-sheevaplug and bsp-sheevaplug-next) or
maybe there are two different machines (e.g. sheevaplug and
sheevaplug-next). There are probably other solutions as well.

- avoid unneeded .inc files. In case there is only one recipe they
only complicate things as people need to examine two files. Of course
if there are multiple recipes for a package they have some merits

- avoid complex and/or unneeded python constructs in recipes. Ideally
a recipe is a description of metadata and it should be able to
describe it using variables only. In practice regularly functions like
do_compile() will be needed though, but it is probably less desirable
to write a complex piece of anonymous python to e.g. populate a -dev
package. W.r.t. readability, maintainability and understandability it
is probably better to just write it out. The goal of a recipe is not
to show off how well-versed one is in python, the goal should be to
come up with something that Joe or Jane Average also can understand.

- strong directory naming for patches. E.g. for foo, patches only for
1.2 should go in foo-1.2/, patches for foo 1.x should go to foo-1/ if
that is possible or otherwise in foo/. Patches should only go in
files/ if they are intended for multiple recipes where the recipes
have different names (e.g. if foo and foobar recipes exist in the same
dir and they need the same foopatch that one could live in files/

- no FILESPATHPKG to different version of a recipe. E.g. in the past
we had the glibc_2.x recipe pointing to the patch dir of 2.y (forgot
the actual values of x and y). If patches are for multiple recipes
they should go in a common higher level named dir (e.g. in the case
above glibc-2 or glibc or maybe the dir can be named differently and
FILESPATHPKG being used. Having patches live in other dirs where it is
not obvious is error prone when recipes are obsoleted.


These points were the things that at this moment popped up in my mind.
There are undoubtly more (e.g order of metadata, mandatory metadata).
It seems useful to agree on where we want to go to before actually
starting to walk (or run :-) ) in an undefined direction and waste
effort

Feel free to add and discuss! Frans.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-17 19:01 ` Frans Meulenbroeks
@ 2011-01-18  7:21   ` Graeme Gregory
  2011-01-18  8:05     ` Otavio Salvador
  0 siblings, 1 reply; 46+ messages in thread
From: Graeme Gregory @ 2011-01-18  7:21 UTC (permalink / raw)
  To: openembedded-devel; +Cc: openembedded-members

On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>
> - where possible stick to one recipe per package. This reduces the
> maintenance work and reduces the QA nightmare of lots of different
> permutations.
> I feel one recipe per package should be the common case for
> applications, and preferably also for libs (although I am well aware
> that especially in the latter case multiple versions cannot always be
> avoided).

OE is not a distro so this is a non starter already, please don't bog
down this discussion by re-opening this again. Angstrom 2008, Angstrom
2010, kaelios and slugos are all released distributions with different
versions of apps just as a starter and they arent even near the total
number of distros in OE.

Graeme




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  7:21   ` Graeme Gregory
@ 2011-01-18  8:05     ` Otavio Salvador
  2011-01-18  8:47       ` Koen Kooi
                         ` (3 more replies)
  0 siblings, 4 replies; 46+ messages in thread
From: Otavio Salvador @ 2011-01-18  8:05 UTC (permalink / raw)
  To: openembedded-devel; +Cc: openembedded-members

On Tue, Jan 18, 2011 at 05:21, Graeme Gregory <dp@xora.org.uk> wrote:
> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>> - where possible stick to one recipe per package. This reduces the
>> maintenance work and reduces the QA nightmare of lots of different
>> permutations.
>> I feel one recipe per package should be the common case for
>> applications, and preferably also for libs (although I am well aware
>> that especially in the latter case multiple versions cannot always be
>> avoided).
>
> OE is not a distro so this is a non starter already, please don't bog
> down this discussion by re-opening this again. Angstrom 2008, Angstrom
> 2010, kaelios and slugos are all released distributions with different
> versions of apps just as a starter and they arent even near the total
> number of distros in OE.

I disagree. I think having too many versions of a package just makes
difficult to get things done:

 - it increases the amount of maintainence work;
 - has a bigger time to get bugs spoted;

Users of old distros ought to use a specific repository and branch.
Master ought to be kept clean for 'next distro release'.

My 2c.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  8:05     ` Otavio Salvador
@ 2011-01-18  8:47       ` Koen Kooi
  2011-01-18  9:17         ` Richard Purdie
  2011-01-18  9:12       ` Graeme Gregory
                         ` (2 subsequent siblings)
  3 siblings, 1 reply; 46+ messages in thread
From: Koen Kooi @ 2011-01-18  8:47 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel


Op 18 jan 2011, om 09:05 heeft Otavio Salvador het volgende geschreven:

> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory <dp@xora.org.uk> wrote:
>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>> - where possible stick to one recipe per package. This reduces the
>>> maintenance work and reduces the QA nightmare of lots of different
>>> permutations.
>>> I feel one recipe per package should be the common case for
>>> applications, and preferably also for libs (although I am well aware
>>> that especially in the latter case multiple versions cannot always be
>>> avoided).
>> 
>> OE is not a distro so this is a non starter already, please don't bog
>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>> 2010, kaelios and slugos are all released distributions with different
>> versions of apps just as a starter and they arent even near the total
>> number of distros in OE.
> 
> I disagree. I think having too many versions of a package just makes
> difficult to get things done:
> 
> - it increases the amount of maintainence work;
> - has a bigger time to get bugs spoted;
> 
> Users of old distros ought to use a specific repository and branch.
> Master ought to be kept clean for 'next distro release'.

I've been using yocto for a few months and I *&#(@$*$(*$($@#($@ hate their 1 version per recipe policy. Every monday I have a working image and every wednesday I have to rebuild from scratch and reevaluate because various things got removed and "upgraded". And since my distro pin file is in a layer, pinnings don't get noticed.
So instead of "building on top" of the metadata I need to resort to "fork" the metadata. And I'm not the only one seeing this problem, the yocto autobuilder is almost perputally red :(

Having 10 versions is too much, but having 3 (oldstable, stable, development) shouldn't be a problem, especially if they use a common .inc file.

Just look at glib-2.0 in yocto. There's only one version, and that's 2.27.5, which is a development release (odd minor number). That's not what I want to use in a distro I need to support, especially looking at the huge API changes going into the 2.27 series.

Similar thing for QT, I don't want 4.6.x to suddenly disappear and be forced to use 4.7.x. I also don't want to move 4.6.x to a distro layer, since QT is way too huge to support on your own.

Or eglibc or gcc or binuils

regards,

Koen


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  8:05     ` Otavio Salvador
  2011-01-18  8:47       ` Koen Kooi
@ 2011-01-18  9:12       ` Graeme Gregory
  2011-01-18 10:15         ` Richard Purdie
  2011-01-18 11:31       ` Mark Brown
  2011-01-18 16:54       ` Tom Rini
  3 siblings, 1 reply; 46+ messages in thread
From: Graeme Gregory @ 2011-01-18  9:12 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel

On 18/01/2011 08:05, Otavio Salvador wrote:
> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>  wrote:
>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>> - where possible stick to one recipe per package. This reduces the
>>> maintenance work and reduces the QA nightmare of lots of different
>>> permutations.
>>> I feel one recipe per package should be the common case for
>>> applications, and preferably also for libs (although I am well aware
>>> that especially in the latter case multiple versions cannot always be
>>> avoided).
>> OE is not a distro so this is a non starter already, please don't bog
>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>> 2010, kaelios and slugos are all released distributions with different
>> versions of apps just as a starter and they arent even near the total
>> number of distros in OE.
> I disagree. I think having too many versions of a package just makes
> difficult to get things done:
>
>   - it increases the amount of maintainence work;
>   - has a bigger time to get bugs spoted;
>
> Users of old distros ought to use a specific repository and branch.
> Master ought to be kept clean for 'next distro release'.
>
>
The ultimate end to this way of thinking is that OE reduces to becoming 
OE-core only as anything else in there is going to be useless to 
multiple distros.

This then leads to the big distros like Angstrom maintaining their own 
large meta data which is basically a fork of todays OE. Then comes the 
point if Angstrom is providing a much better service to other DISTROS 
like SlugOS than OE-core does then they might save themselves 
maintenance time by re-using our large data set. Basically destroying 
the value of OE and forcing a fork.

People really have to get out of the mindset that OE is a DISTRO, it is 
not and must provide services to all DISTRO.

Ill also point out that gentoo which is probably closest to OE in 
management of metadata does not pathalogically delete old versions.

Graeme




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  8:47       ` Koen Kooi
@ 2011-01-18  9:17         ` Richard Purdie
  0 siblings, 0 replies; 46+ messages in thread
From: Richard Purdie @ 2011-01-18  9:17 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel

On Tue, 2011-01-18 at 09:47 +0100, Koen Kooi wrote:
> Op 18 jan 2011, om 09:05 heeft Otavio Salvador het volgende geschreven:
> > Users of old distros ought to use a specific repository and branch.
> > Master ought to be kept clean for 'next distro release'.
> 
> I've been using yocto for a few months and I *&#(@$*$(*$($@#($@ hate
> their 1 version per recipe policy. Every monday I have a working image
> and every wednesday I have to rebuild from scratch and reevaluate
> because various things got removed and "upgraded". And since my distro
> pin file is in a layer, pinnings don't get noticed.
> So instead of "building on top" of the metadata I need to resort to
> "fork" the metadata. And I'm not the only one seeing this problem, the
> yocto autobuilder is almost perputally red :(

You will notice a change in behaviour from Feb 4th as we hit the release
"freeze" for Yocto's next release as per the schedule on the wiki. At
the moment the codebase is in development flux, then we go to the
stabilisation window. There is a well defined schedule for this and
everyone should have known expectations of what is happening.

The autobuilder has been a bit rough I agree. Saul correctly did call
for stabilisation of the known issues, we've done that, hit green builds
and because the freeze is coming and we've still some major work to
merge in, we're continuing to develop. I appreciated Saul's work in that
area.

So you can see that whilst we are making changes, we are also testing
what we're doing and fixing the regressions we can find. If that testing
is missing things that cause people problems, I'd suggest helping us
find new tests that cover them too.

> Having 10 versions is too much, but having 3 (oldstable, stable,
> development) shouldn't be a problem, especially if they use a
> common .inc file.
> 
> Just look at glib-2.0 in yocto. There's only one version, and that's
> 2.27.5, which is a development release (odd minor number). That's not
> what I want to use in a distro I need to support, especially looking
> at the huge API changes going into the 2.27 series.

Hmm, we should not be using an odd minor number gnome release component
I agree. That should not have happened something has gone wrong there.

> Similar thing for QT, I don't want 4.6.x to suddenly disappear and be
> forced to use 4.7.x. I also don't want to move 4.6.x to a distro
> layer, since QT is way too huge to support on your own.
> 
> Or eglibc or gcc or binuils

For components like these, Yocto aims to pick a version and run with
that for a given release. We want to try and stay reasonable current as
long as the version works. For something based on Yocto, it may be that
as the core moves forward, older versions move into layers if they're
still needed.

Yocto isn't going to make everyone happy or do everything right straight
out the box as we're learning. What I do intend to do is not be afraid
to try things differently but listen to the feedback, adapting
accordingly. OpenEmbedded is stagnating by comparison and I don't think
its healthy.

Cheers,

Richard




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  9:12       ` Graeme Gregory
@ 2011-01-18 10:15         ` Richard Purdie
  2011-01-18 11:05           ` Frans Meulenbroeks
  0 siblings, 1 reply; 46+ messages in thread
From: Richard Purdie @ 2011-01-18 10:15 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel

On Tue, 2011-01-18 at 09:12 +0000, Graeme Gregory wrote:
> On 18/01/2011 08:05, Otavio Salvador wrote:
> > On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>  wrote:
> >> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
> >>> - where possible stick to one recipe per package. This reduces the
> >>> maintenance work and reduces the QA nightmare of lots of different
> >>> permutations.
> >>> I feel one recipe per package should be the common case for
> >>> applications, and preferably also for libs (although I am well aware
> >>> that especially in the latter case multiple versions cannot always be
> >>> avoided).
> >> OE is not a distro so this is a non starter already, please don't bog
> >> down this discussion by re-opening this again. Angstrom 2008, Angstrom
> >> 2010, kaelios and slugos are all released distributions with different
> >> versions of apps just as a starter and they arent even near the total
> >> number of distros in OE.
> > I disagree. I think having too many versions of a package just makes
> > difficult to get things done:
> >
> >   - it increases the amount of maintainence work;
> >   - has a bigger time to get bugs spoted;
> >
> > Users of old distros ought to use a specific repository and branch.
> > Master ought to be kept clean for 'next distro release'.
> >
> >
> The ultimate end to this way of thinking is that OE reduces to becoming 
> OE-core only as anything else in there is going to be useless to 
> multiple distros.
> 
> This then leads to the big distros like Angstrom maintaining their own 
> large meta data which is basically a fork of todays OE. Then comes the 
> point if Angstrom is providing a much better service to other DISTROS 
> like SlugOS than OE-core does then they might save themselves 
> maintenance time by re-using our large data set. Basically destroying 
> the value of OE and forcing a fork.
> 
> People really have to get out of the mindset that OE is a DISTRO, it is 
> not and must provide services to all DISTRO.
> 
> Ill also point out that gentoo which is probably closest to OE in 
> management of metadata does not pathalogically delete old versions.

Let me just clearly differentiate between the core layer we're trying to
create and meta-oe, the latter of which may contain different layers
with different focuses.

I think the policy in the core can differ from that in meta-oe, it fact
it has to or they are the same thing as Graeme points out.

Different distros have different needs and meta-oe needs to support them
as OE does today. 

The idea is the high quality and testing of the core allows OE to move
forward technologically and gives a shared known quantity for people to
base things off. By known quantity, I mean that people can see and
expect a certain level of tests and functionality to work from it. If
for whatever reason those core versions aren't good enough for a distro
then the distros can override and they can pool their knowledge about
those overrides. If it appears they have to override more often that
they don't, I'd suggest the core isn't fulfilling people's needs and we
should fix the core, testing or processes leading to that. Only time and
experimentation will show exactly how this will work out but I'm pretty
sure it can be done.

I'm certainly planning to learn what difficulties Koen is having with
Yocto and see what we can do to address them. I'd also like to thank
Koen for his efforts in working through this as its only by doing that
we're going to learn and move forward.

Cheers,

Richard





^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 10:15         ` Richard Purdie
@ 2011-01-18 11:05           ` Frans Meulenbroeks
  0 siblings, 0 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-18 11:05 UTC (permalink / raw)
  To: openembedded-members, openembedded-devel

We had a discussion on #yocto on this.
Per Graeme's request I am posting a log of that conversation.
The interesting parts start probably from 10.45 to 11.05 and from 11.37 onwards.
However I did not want to edit the log, so I left it all in.

Frans.

(09:46:13 AM) eFfeM_work: RP__ is there a policy in yocto wrt multiple
recipes for a package (e.g. for two different versions); I seem to
recall you mentioned in the past typically one version, but I cannot
really find the ref any more
(09:46:23 AM) RP__: yuke: I might be able to help with that but I
don't want to duplicate anything you're doing
(09:46:58 AM) RP__: eFfeM_work: Yes, there is a strong pressure for
one version (or a stable and a development, maybe scm based recipe)
(09:46:59 AM) yuke: RP__: sure, i will send out the V2 patch
(09:47:36 AM) eFfeM_work: RP__ thanks that is what I recall from the
past too, is there a ref to a written policy statement on this ?
(09:49:54 AM) RP__: eFfeM_work: Its been stated publicly enough but I
think we should have something on the wiki
RP__ rphillips
(09:50:07 AM) eFfeM_work: RP__ thanks
(09:50:31 AM) jiajun1 left the room (quit: Remote host closed the connection).
(10:02:09 AM) bluelightning left the room (quit: Quit: Konversation
terminated!!!111).
(10:02:24 AM) bluelightning [~paul@83.217.123.106] entered the room.
(10:02:31 AM) bluelightning left the room (quit: Changing host).
(10:02:31 AM) bluelightning
[~paul@pdpc/supporter/professional/bluelightning] entered the room.
(10:06:12 AM) KevinTian left the room.
(10:15:11 AM) bluelightning: morning all
(10:16:31 AM) RP__: hi bluelightning
(10:20:12 AM) XorA: morning
(10:27:20 AM) RP__: hi XorA
(10:28:56 AM) XorA: finally some real juicy yocto discussion in OE :-D
(10:40:20 AM) florian_kc [~fuchs@Maemo/community/contributor/florian]
entered the room.
(10:41:20 AM) florian_kc is now known as florian
(10:42:49 AM) Jay7: RP__, XorA: is Yocto one-distro-framework?
(10:43:21 AM) Jay7: it looks alike :)
(10:43:36 AM) Jay7: Yocto or poky even
(10:45:26 AM) XorA: traditionally poky always built poky, but I guess
RP can answer officially
(10:48:11 AM) XorA: the model I have in my head for
Yocto/OE-core/OE-universe is having a core of tested stable toolchains
which will as a neccessity have different versions of gcc/glibc/uclibc
the meta-oe building on this to support multi distros in the same way
we do now
(10:48:21 AM) RP__: Jay7: No, its not indented to be one distro
(10:48:56 AM) Jay7: RP__: but what is current situation? Have yocto or
poky other distros?
(10:49:14 AM) XorA: but people requiring exactly one version
pathalogically means pretty much OE becomes a DISTRO and we kick out
well respected distros like slugos which is a backwards step
(10:49:20 AM) RP__: Jay7: Koen is trying an experiment with Angstrom
(10:50:33 AM) RP__: I don't think Yocto is saying one version
pathologically, but there is a pressure there to try and reduce the
numbers of versions to a small set of well tested ones rather than
many different combinations
(10:50:53 AM) Jay7: so, we are trying to pull
multiple-distro-building-framework on top of framework which is
developed with one distro in mind currently? :)
(10:51:11 AM) XorA: RP__: minimal subset is fine, but certain peoples
crusade to reduce to just one version is wrong
(10:51:32 AM) Jay7: are all Yocto devels know that Yocto should
support multiple distros? :)
(10:51:33 AM) RP__: Jay7: Its not maintained with one distro in mind,
its maintained to try and create a core we can test
(10:51:55 AM) RP__: Jay7: Its been mentioned to them
(10:52:30 AM) Jay7: well, then things are not so bad :)
(10:52:46 AM) RP__: XorA: One version for everything is not realistic
(10:53:15 AM) RP__: Jay7: remember that Yocto has various commercial
interests wanting to build on top of it
(10:53:31 AM) XorA: RP__: way I see it if a version is locked in a
DISTRO then that distro has chosen for a reason to use that version
and we need to maintain that metadata or agree with distro an upgrade
path
(10:54:07 AM) RP__: XorA: Right, the tricky bit is going to be finding
out who is doing that
(10:54:21 AM) XorA: I think sometimes people forget Angstrom and
SlugOS are actually already released distros in the same way debian is
(10:54:55 AM) XorA: and SHR for that matter which has another large user base
(10:55:24 AM) RP__: XorA: I suspect released distros like that would
be best working on a release version/branch of the yocto core in this
model
(10:56:07 AM) RP__: XorA: When they come to upgrade to the next core
release, they make a decision whether to take package upgrades or not.
Not would mean add the version they want to their layer
(10:56:08 AM) eFfeM_work: XorA: I feel that if a distro locks a
recipe, it becomes the responsibility of the distro to maintain that
specific version, not from the recipe maintainer
(10:56:18 AM) XorA: RP__: I dont think its the yocto core that is an
issue, its meta-oe
(10:56:23 AM) eFfeM_work: if the distro pins, the distro should bear
the consequences not the recipe maintainer
(10:56:28 AM) RP__: XorA: right
(10:57:19 AM) eFfeM_work: btw with a release model this should be
simpler, then you could have distro-stable or whatever which builds
upon yocto releases
(10:57:48 AM) XorA: its all very well using branches of meta-oe but
experience has shown us fixes never get back to mainline
(10:57:59 AM) RP__: eFfeM_work: I think you're making the issue very
black and white but there is grey in there too
RP__ rphillips
(10:58:18 AM) Jay7: XorA: add JLime to your list of most used distros :)
(10:58:33 AM) eFfeM_work: RP__: sure there is grey
(10:58:45 AM) XorA: Jay7: exactly, there are lots more than my brain
can hold, and the one recipe model just doesnt work
(10:59:13 AM) eFfeM_work: see also my mail; for apps I'd say
preferably one version for libs there are definitely cases where more
versions are desired
(10:59:19 AM) eFfeM_work: and for toolchains it is unavoidable
(10:59:23 AM) eFfeM_work: meeting, back later
(10:59:43 AM) XorA: eFfeM_work: that is unworkable, simple as that
(11:01:34 AM) XorA: once again this issue is going to obscure real
discussion of yocto/OE working together :-(
(11:02:24 AM) Jay7: at least there is some constructive
(11:02:52 AM) XorA: well at least every path seems to lead to
Yocto/OE-core becoming one
(11:03:10 AM) XorA: which means more QA on the basics
(11:03:28 AM) XorA: we can fight about meta-oe all we like without
affecting that
(11:03:34 AM) XorA: effecting
(11:03:40 AM) XorA: or maybe some other word
(11:03:47 AM) ***XorA is too hung over for english
(11:04:08 AM) Jay7: don't forget, while we are talking Koen is working :)
(11:04:54 AM) XorA: as of weds I plan to join koen on meta-oe work,
Ive been prevented so far by being on holiday for first time in 2
years
(11:05:31 AM) Jay7: I'll join to testing after major kexecboot release :)
(11:08:07 AM) Jay7: XorA: what do you think about OE infrastructure
(e.g tinderbox)? is there any value to improve it now?
(11:08:30 AM) XorA: Jay7: I havent actually used the tinderbox at all
(11:08:32 AM) Jay7: yocto is using buildbot for similar purposes
(11:08:54 AM) Jay7: well.. I'll ask differently
(11:09:14 AM) XorA: I think we should ask Yocto nicely for CPU power
for things like buildbot, OE cant really afford that sort of grunt
sitting in a data center
(11:09:36 AM) Jay7: how long will take migration to Yocto layer? :)
(11:09:49 AM) XorA: Jay7: how long is that string again :-D
(11:12:59 AM) Jay7: good thing behind tinderbox is oe-stats
(11:13:13 AM) Jay7: users may build at home but report status to single place
(11:13:24 AM) Jay7: not sure about yocto's buildbot
(11:15:55 AM) incandescant [~joshual@83.217.123.106] entered the room.
(11:16:08 AM) XorA: I think they are different use cases, that is
useful for users to report issues they see, buildbot is useful for
continuous integration, I think both should be done in ideal world
(11:16:25 AM) RP__: XorA: I have to admit that Yocto is also under
resourced for CPU power in data centres at the moment
(11:16:28 AM) ***XorA fixes issues with public keys
(11:16:32 AM) RP__: XorA: We're working on it though
(11:16:53 AM) RP__: and yes, tinderbox and buildbot fulfil different roles
(11:17:20 AM) XorA: RP__: still is an area why I think we can see OE
would benifit from some form of support in this area, but we can wait
until Yocto had its own infrastructure issues sorted
(11:17:58 AM) RP__: XorA: The trouble is our testcases are expanding
more rapidly than the hardware :)
(11:18:01 AM) ***XorA doesnt expect solutions to happen all tomorrow,
we know the real world doesnt run at RP__ speed :-D
(11:18:25 AM) RP__: On the plus side, this means we have tests being
developed and run
(11:18:34 AM) RP__: e.g. boot testing of images at build time
(11:19:35 AM) RP__: We managed to borrow a 40 way machine to iron out
some of the parallelism bugs in Yocto :)
(11:19:41 AM) XorA: nice
(11:19:59 AM) RP__: I don't think we want to give that back...
(11:20:01 AM) ***XorA has two 24 ways machine here, but they run RHEL4
so getting OE to run might be an issue
(11:20:31 AM) RP__: XorA: Yocto should run there with the standalone
python tarball
(11:21:06 AM) Jay7: XorA: use lxc to run OE in container ;)
(11:21:09 AM) XorA: RP__: I might negotiate with my boss for some CPU
time then, one of the machines certainly stands idle a lot
(11:21:30 AM) XorA: Jay7: lxc?
(11:21:37 AM) ***Jay7 have 6-core on desk.. should load it with something useful
(11:21:44 AM) Jay7: XorA: linux containers
(11:21:51 AM) Jay7: mainlined to kernel
(11:21:53 AM) XorA: Jay7: link me in :-D
(11:22:05 AM) Jay7: I'm building inside lxc now
(11:22:07 AM) XorA: Jay7: in kernel as old as RHEL4?
(11:22:17 AM) Jay7: XorA: oh.. rhel4.. :)
(11:22:36 AM) XorA: nice rock solid OS, but not exactly new fangled
(11:22:38 AM) Jay7: not sure, but debian chroot may help then ;)
(11:23:01 AM) RP__: Jay7: Its the ancient kernel you can't change
(11:23:20 AM) XorA: although debian lenny should operate ok in a chroot on that
(11:23:35 AM) RP__: XorA: maybe :)
(11:23:38 AM) XorA: if not the previous debian stable certainly
(11:23:48 AM) ***RP__ likes the standalone python approach
(11:24:07 AM) ***XorA actually has SRPMS for a python upgrade rolled
(11:24:18 AM) XorA: that allow a parrelel install of python
(11:24:40 AM) XorA: RPM has some nice features that made that easy to do
(11:25:42 AM) RP__: The rpm backend we now have is certainly
interesting in yocto too
(11:36:21 AM) eFfeM_work: re
(11:37:57 AM) eFfeM_work: wrt: (10:59:43 AM) XorA: eFfeM_work: that is
unworkable, simple as that
(11:38:49 AM) eFfeM_work: multiple versions are a PITA wrt
maintenance; note also that I am well aware that one version is not
always feasible, that is why i wrote "should be the common case"
(11:39:55 AM) eFfeM_work: one of the issues we are facing is that
angstrom is using git head to release, and git head is bleeding edge
so sometimes you bleed
(11:40:40 AM) eFfeM_work: I can imagine a distro basing itself on
releases and a -next version for git head (somewhat like the debian
model)
(11:43:15 AM) XorA: eFfeM_work: you miss all the subtle issues of
distro maintenance, say I have a distro for devices with limited nand,
I use abiword versions XXX which fits, new version has many new
features that are cool but increases footprint 2M I want to stay with
old version as new doesnt fit. I dont want to be fighting you every
two weeks to keep old version
(11:43:32 AM) eFfeM_work: btw and wrt the red on the autobuilder
remark, i feel this is a process issue at the yocto side
(11:45:29 AM) eFfeM_work: XorA, I understand that, guess in case of
multiple versions we should record why we have multiple versions and
who is responsible for that; I know the footprint issue also exists
for pango or cairo
(11:46:12 AM) eFfeM_work: the other side is that you get lots of
versions like we have now and the status and support for the versions
is vague for most of them
(11:46:25 AM) XorA: eFfeM_work: we want the policy to suggest a
minimal set of needed recipes and some way to mark in use ones, I see
we dont want to keep ALL versions, but we dont want to be forcing
people down to just one version
(11:47:35 AM) XorA: bitbake can help here as it knows which versions
have been "locked"
(11:47:59 AM) XorA: I wrote some tasks for openmoko once to do this
sort of data processing
(11:48:47 AM) eFfeM_work: XorA, I understand the policy part,
unfortunately at the moment we do not have the removal part of the
policy.
(11:48:59 AM) KevinTian [~ktian1@nat/intel/x-grijessmqauiwiok] entered the room.
(11:49:24 AM) XorA: eFfeM_work: my suggestion would be a cronjob that
crunches the data once an X timeperiod and policy of removal if no-one
objects to the removal report
(11:50:03 AM) eFfeM_work: which makes we keep all kind of old stuff
around because some maybe-not-even-used-any-more distro locks them
(11:50:35 AM) XorA: eFfeM_work: if its really not used "and we can
check by sending emails to OE-dev" then removing that distro from OE
will auto remove old versions
(11:50:37 AM) qhe left the room (quit: Ping timeout: 240 seconds).
(11:50:53 AM) XorA: eFfeM_work: maybe a requirement that all active
distros have a contactable maintainer
(11:51:38 AM) XorA: I dont think its too much to ask to maintain a
current email address in your distro
(11:51:41 AM) eFfeM_work: at some point if a distro does not want to
move forward it should take up maintenance responsibility of such
recipes (and perhaps that is the time to move them to the distro
layer). I happen to have some old recipes in my own overlay because I
need these
(11:52:09 AM) XorA: again that is something that can be done with
discussion with distro maintainer
(11:52:29 AM) XorA: certainly something TSC can drive forward
(11:52:52 AM) XorA: if TSC feel a distro has entered steady state then
they can open dialogue to move it to branch/tag
(11:53:03 AM) eFfeM_work: XorA: I fully agree with contactable distro
maintainers, I proposed this half a year ago or so, but failed
miserably. I could not identify a maintainer for several distros, but
there were also forces against removal of those what I call stale
distro's.
(11:53:48 AM) XorA: eFfeM_work: obvious course of action at this
junction is to requre maintainers to move their own distro to meta-oe
(11:53:58 AM) XorA: eFfeM_work: obviously the ones who dont read email
wont port over
(11:54:10 AM) XorA: eFfeM_work: but wont lose anything as old OE will
still exist
(11:54:11 AM) eFfeM_work: yes and that will at least solve part of the problem
(11:54:40 AM) XorA: eFfeM_work: fancy pasting this log into an email,
this is constructive, but Im on an EEE
(11:56:33 AM) eFfeM_work: btw despite popular belief I am not against
versions, as long as it is clear who is repsonsible for a version (and
probably why there are multiple versions). I never suggested removing
gcc recipes as khem takes good care of them, but in other cases people
seem to create new versions and just leave an unmaintained trail of
junk (which I would like to avoid in the future)
(11:56:52 AM) eFfeM_work: XorA: Do you want that mail to you or to the list
(11:57:09 AM) XorA: eFfeM_work: on the list, get others involved who
are not awake yet
(11:57:16 AM) eFfeM_work: XorA: will do



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  8:05     ` Otavio Salvador
  2011-01-18  8:47       ` Koen Kooi
  2011-01-18  9:12       ` Graeme Gregory
@ 2011-01-18 11:31       ` Mark Brown
  2011-01-18 18:45         ` Frans Meulenbroeks
  2011-01-18 16:54       ` Tom Rini
  3 siblings, 1 reply; 46+ messages in thread
From: Mark Brown @ 2011-01-18 11:31 UTC (permalink / raw)
  To: openembedded-members; +Cc: openembedded-devel

On Tue, Jan 18, 2011 at 06:05:41AM -0200, Otavio Salvador wrote:
> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory <dp@xora.org.uk> wrote:

> > OE is not a distro so this is a non starter already, please don't bog
> > down this discussion by re-opening this again. Angstrom 2008, Angstrom
> > 2010, kaelios and slugos are all released distributions with different
> > versions of apps just as a starter and they arent even near the total
> > number of distros in OE.

> I disagree. I think having too many versions of a package just makes
> difficult to get things done:

>  - it increases the amount of maintainence work;
>  - has a bigger time to get bugs spoted;

> Users of old distros ought to use a specific repository and branch.
> Master ought to be kept clean for 'next distro release'.

I don't think there's a massive conflict between the idea that it's a
good idea to keep the number of versions that are maintained limited and
the idea that the limit on the number of versions being maintained
should be greater than one which seems to be all that's being said here.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18  8:05     ` Otavio Salvador
                         ` (2 preceding siblings ...)
  2011-01-18 11:31       ` Mark Brown
@ 2011-01-18 16:54       ` Tom Rini
  2011-01-18 20:12         ` Koen Kooi
  3 siblings, 1 reply; 46+ messages in thread
From: Tom Rini @ 2011-01-18 16:54 UTC (permalink / raw)
  To: openembedded-devel

On 01/18/2011 01:05 AM, Otavio Salvador wrote:
> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>  wrote:
>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>> - where possible stick to one recipe per package. This reduces the
>>> maintenance work and reduces the QA nightmare of lots of different
>>> permutations.
>>> I feel one recipe per package should be the common case for
>>> applications, and preferably also for libs (although I am well aware
>>> that especially in the latter case multiple versions cannot always be
>>> avoided).
>>
>> OE is not a distro so this is a non starter already, please don't bog
>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>> 2010, kaelios and slugos are all released distributions with different
>> versions of apps just as a starter and they arent even near the total
>> number of distros in OE.
>
> I disagree. I think having too many versions of a package just makes
> difficult to get things done:
>
>   - it increases the amount of maintainence work;
>   - has a bigger time to get bugs spoted;
>
> Users of old distros ought to use a specific repository and branch.
> Master ought to be kept clean for 'next distro release'.

I agree, at least going forward.  We must make it easier for 
distributions to say "here is my 'stable' release" and "here is my 
development release".

First, I'm not picking on Angstrom here, really, I swear.  It's just a 
good example.

But we also don't want to be unreasonable or unbending here.  We'll have 
to have multiple udevs (due to having different kernel versions as some 
HW isn't on the latest and greatest).  And if DistroA says they really 
want to stick to busybox 1.17.4 for a while, we should let that happen 
too.  But I don't think we want to have to carry on the recipes that 
angstrom-2008.1 wants and angstrom-2010.x wants and angstrom-2011.x 
wants and angstrom-2012.x want into 2013, in master.

For example, at some point we want to switch to libtool 2.4 only.  And 
that would certainly be a headache for angstrom-2008.1 (but we're glad, 
really! for angstrom-2010.x using 2.4 and testing and fixing things). 
So wouldn't it be a good thing to be able to say that if you want 
angstrom-2008.1 you do ... this ... and get the layers that give a good 
stable 2008.1, based on whatever policy Angstrom wants for doing updates?

-- 
Tom Rini
Mentor Graphics Corporation



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 11:31       ` Mark Brown
@ 2011-01-18 18:45         ` Frans Meulenbroeks
  0 siblings, 0 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-18 18:45 UTC (permalink / raw)
  To: openembedded-members, openembedded-devel

2011/1/18 Mark Brown <broonie@sirena.org.uk>:
> On Tue, Jan 18, 2011 at 06:05:41AM -0200, Otavio Salvador wrote:
>> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory <dp@xora.org.uk> wrote:
>
>> > OE is not a distro so this is a non starter already, please don't bog
>> > down this discussion by re-opening this again. Angstrom 2008, Angstrom
>> > 2010, kaelios and slugos are all released distributions with different
>> > versions of apps just as a starter and they arent even near the total
>> > number of distros in OE.
>
>> I disagree. I think having too many versions of a package just makes
>> difficult to get things done:
>
>>  - it increases the amount of maintainence work;
>>  - has a bigger time to get bugs spoted;
>
>> Users of old distros ought to use a specific repository and branch.
>> Master ought to be kept clean for 'next distro release'.
>
> I don't think there's a massive conflict between the idea that it's a
> good idea to keep the number of versions that are maintained limited and
> the idea that the limit on the number of versions being maintained
> should be greater than one which seems to be all that's being said here.

To make things a little bit clearer.
I am not opposed to multiple versions per se.
I have a concern wrt maintenance though, and that is one of the
reasons I prefer a limited number of recipes.

As a recipe maintainer I feel that I do not need to carry the burden
caused by a decision of a distro for a specific version of a recipe.
E.g. if I maintain package foo, and that package is at version X and
upstream moves to version Y, I think it is part of the responsibility
of the recipe maintainer to move to Y (after testing of course).
However if Y is proven to be stable, generally as a maintainer I want
to retire X (unless there are compelling reasons to keep it, e.g.
because heavily changed functionality or incompatible api's or so).
If a distro feels it wants to stay at X, I have no problem with that,
but I feel that distro should also take over the maintenance
responsibility (and retire X if it is not used any more). Don't put
the burden of your choices onto someone else!
I have neither the time or the interest to support a substantial
number of variants that IMHO are outdated. If someone else feels (s)he
wants to keep using them, feel free, but then also please take the
maintenance burden.

Unfortunately some of our developers have a habit of creating lots of
new recipes but rarely clean up (but also do not maintain the old
stuff), leaving a trail of undefined/orphaned recipes.
I think we should try to avoid having those orphaned recipes.

Wrt the example of Graeme on abiword (not sure if it was here or on
irc). His reasoning was that he wanted to keep an older version
because it was much smaller. Being in the business of resource
constrained embedded systems I clearly understand this. However I feel
it is better to make that explicit, e.g. by introducing a recipe
foo-small_X.bb adjacent to foo_Y.bb (or maybe even foo_X.bb), than to
have it implicit (like in the case foo_X.bb and foo_Y.bb).
In a few cases we already do so, but typically then it is
function/interface driven. bluez vz bluez4 comes to mind.

Thinking of it, we could also go for a "previous" layer or so where we
move older versions to. Those who want them then can add that layer,
those who do not want them do not include that layer.

Frans.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 16:54       ` Tom Rini
@ 2011-01-18 20:12         ` Koen Kooi
  2011-01-18 20:48           ` Tom Rini
  2011-01-19  7:47           ` Frans Meulenbroeks
  0 siblings, 2 replies; 46+ messages in thread
From: Koen Kooi @ 2011-01-18 20:12 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 18-01-11 17:54, Tom Rini wrote:
> On 01/18/2011 01:05 AM, Otavio Salvador wrote:
>> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>  wrote:
>>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>>> - where possible stick to one recipe per package. This reduces the
>>>> maintenance work and reduces the QA nightmare of lots of different
>>>> permutations.
>>>> I feel one recipe per package should be the common case for
>>>> applications, and preferably also for libs (although I am well aware
>>>> that especially in the latter case multiple versions cannot always be
>>>> avoided).
>>>
>>> OE is not a distro so this is a non starter already, please don't bog
>>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>>> 2010, kaelios and slugos are all released distributions with different
>>> versions of apps just as a starter and they arent even near the total
>>> number of distros in OE.
>>
>> I disagree. I think having too many versions of a package just makes
>> difficult to get things done:
>>
>>   - it increases the amount of maintainence work;
>>   - has a bigger time to get bugs spoted;
>>
>> Users of old distros ought to use a specific repository and branch.
>> Master ought to be kept clean for 'next distro release'.
> 
> I agree, at least going forward.  We must make it easier for
> distributions to say "here is my 'stable' release" and "here is my
> development release".
> 
> First, I'm not picking on Angstrom here, really, I swear.  It's just a
> good example.
> 
> But we also don't want to be unreasonable or unbending here.  We'll have
> to have multiple udevs (due to having different kernel versions as some
> HW isn't on the latest and greatest).  And if DistroA says they really
> want to stick to busybox 1.17.4 for a while, we should let that happen
> too.  But I don't think we want to have to carry on the recipes that
> angstrom-2008.1 wants and angstrom-2010.x wants and angstrom-2011.x
> wants and angstrom-2012.x want into 2013, in master.

And noone says you should. At some point 2010.x works well enough to
force 2008.1 into hiding and start 201Y.x. The current situation where
the "unstable" 2010.x ended up in a product is largely due to the gcc
people breaking the NEON intrinsics interface API in between 4.3 and 4.5.

> For example, at some point we want to switch to libtool 2.4 only.  And
> that would certainly be a headache for angstrom-2008.1 (but we're glad,
> really! for angstrom-2010.x using 2.4 and testing and fixing things). So
> wouldn't it be a good thing to be able to say that if you want
> angstrom-2008.1 you do ... this ... and get the layers that give a good
> stable 2008.1, based on whatever policy Angstrom wants for doing updates?

In the past the angstrom people created a stable branch and supported
that for a given release. The same can be done in the layering script,
where it would just lock down to certain revisions of various layers.

But in the end if boils down to "Does OE wants to make life hard for
DISTROs or easy". Frans is firmly in the "make it hard" camp, I hope
others have a saner point of view.

If you're forcing 90% of your users to put e.g. udev_162.bb in their
layer you're doing it wrong. But you're also doing it wrong if you have
20 udev recipes :)


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFNNfREMkyGM64RGpERAmM/AJ9G63cPlLZQ/7V47IfNYh9KM4UbPACgkHOE
+G8xtPepz5vtf+Lmoi5ismk=
=lMtI
-----END PGP SIGNATURE-----




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 20:12         ` Koen Kooi
@ 2011-01-18 20:48           ` Tom Rini
  2011-01-18 22:05             ` Koen Kooi
  2011-01-18 22:48             ` Khem Raj
  2011-01-19  7:47           ` Frans Meulenbroeks
  1 sibling, 2 replies; 46+ messages in thread
From: Tom Rini @ 2011-01-18 20:48 UTC (permalink / raw)
  To: openembedded-devel

On 01/18/2011 01:12 PM, Koen Kooi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 18-01-11 17:54, Tom Rini wrote:
>> On 01/18/2011 01:05 AM, Otavio Salvador wrote:
>>> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>   wrote:
>>>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>>>> - where possible stick to one recipe per package. This reduces the
>>>>> maintenance work and reduces the QA nightmare of lots of different
>>>>> permutations.
>>>>> I feel one recipe per package should be the common case for
>>>>> applications, and preferably also for libs (although I am well aware
>>>>> that especially in the latter case multiple versions cannot always be
>>>>> avoided).
>>>>
>>>> OE is not a distro so this is a non starter already, please don't bog
>>>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>>>> 2010, kaelios and slugos are all released distributions with different
>>>> versions of apps just as a starter and they arent even near the total
>>>> number of distros in OE.
>>>
>>> I disagree. I think having too many versions of a package just makes
>>> difficult to get things done:
>>>
>>>    - it increases the amount of maintainence work;
>>>    - has a bigger time to get bugs spoted;
>>>
>>> Users of old distros ought to use a specific repository and branch.
>>> Master ought to be kept clean for 'next distro release'.
>>
>> I agree, at least going forward.  We must make it easier for
>> distributions to say "here is my 'stable' release" and "here is my
>> development release".
>>
>> First, I'm not picking on Angstrom here, really, I swear.  It's just a
>> good example.
>>
>> But we also don't want to be unreasonable or unbending here.  We'll have
>> to have multiple udevs (due to having different kernel versions as some
>> HW isn't on the latest and greatest).  And if DistroA says they really
>> want to stick to busybox 1.17.4 for a while, we should let that happen
>> too.  But I don't think we want to have to carry on the recipes that
>> angstrom-2008.1 wants and angstrom-2010.x wants and angstrom-2011.x
>> wants and angstrom-2012.x want into 2013, in master.
>
> And noone says you should. At some point 2010.x works well enough to
> force 2008.1 into hiding and start 201Y.x. The current situation where
> the "unstable" 2010.x ended up in a product is largely due to the gcc
> people breaking the NEON intrinsics interface API in between 4.3 and 4.5.

ick, I didn't know about that...

>> For example, at some point we want to switch to libtool 2.4 only.  And
>> that would certainly be a headache for angstrom-2008.1 (but we're glad,
>> really! for angstrom-2010.x using 2.4 and testing and fixing things). So
>> wouldn't it be a good thing to be able to say that if you want
>> angstrom-2008.1 you do ... this ... and get the layers that give a good
>> stable 2008.1, based on whatever policy Angstrom wants for doing updates?
>
> In the past the angstrom people created a stable branch and supported
> that for a given release. The same can be done in the layering script,
> where it would just lock down to certain revisions of various layers.

So, I think we agree.  Distros should be saying "if you want our stable 
release you should be over here..." and if you want our development WIP, 
you should be over here.

> But in the end if boils down to "Does OE wants to make life hard for
> DISTROs or easy". Frans is firmly in the "make it hard" camp, I hope
> others have a saner point of view.
>
> If you're forcing 90% of your users to put e.g. udev_162.bb in their
> layer you're doing it wrong. But you're also doing it wrong if you have
> 20 udev recipes :)

I think we also agree here.  But what's the rule of thumb(s) we want to 
have, to provide enough choice without too much headache?  As I said 
elsewhere, .inc files should probably be used a whole lot more, to help 
with the problem of recipe bugfixing and N recipes for an app with the 
problem.  We should probably also say that in addition to the "keep the 
last GPLv2+ version around" rule of thumb we should also have a "keep 
the latest stable release" around too.  But what else?  To use busybox 
as an example, do we really need to keep 1.18.0 and 1.18.1 around when 
we have 1.18.2?  How about if we make the delta between the 3 be just 
the SRC_URI + checksums?

-- 
Tom Rini
Mentor Graphics Corporation



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 20:48           ` Tom Rini
@ 2011-01-18 22:05             ` Koen Kooi
  2011-01-19  8:45               ` Frans Meulenbroeks
  2011-01-18 22:48             ` Khem Raj
  1 sibling, 1 reply; 46+ messages in thread
From: Koen Kooi @ 2011-01-18 22:05 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 18-01-11 21:48, Tom Rini wrote:
> On 01/18/2011 01:12 PM, Koen Kooi wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 18-01-11 17:54, Tom Rini wrote:
>>> On 01/18/2011 01:05 AM, Otavio Salvador wrote:
>>>> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>   wrote:
>>>>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>>>>> - where possible stick to one recipe per package. This reduces the
>>>>>> maintenance work and reduces the QA nightmare of lots of different
>>>>>> permutations.
>>>>>> I feel one recipe per package should be the common case for
>>>>>> applications, and preferably also for libs (although I am well aware
>>>>>> that especially in the latter case multiple versions cannot always be
>>>>>> avoided).
>>>>>
>>>>> OE is not a distro so this is a non starter already, please don't bog
>>>>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>>>>> 2010, kaelios and slugos are all released distributions with different
>>>>> versions of apps just as a starter and they arent even near the total
>>>>> number of distros in OE.
>>>>
>>>> I disagree. I think having too many versions of a package just makes
>>>> difficult to get things done:
>>>>
>>>>    - it increases the amount of maintainence work;
>>>>    - has a bigger time to get bugs spoted;
>>>>
>>>> Users of old distros ought to use a specific repository and branch.
>>>> Master ought to be kept clean for 'next distro release'.
>>>
>>> I agree, at least going forward.  We must make it easier for
>>> distributions to say "here is my 'stable' release" and "here is my
>>> development release".
>>>
>>> First, I'm not picking on Angstrom here, really, I swear.  It's just a
>>> good example.
>>>
>>> But we also don't want to be unreasonable or unbending here.  We'll have
>>> to have multiple udevs (due to having different kernel versions as some
>>> HW isn't on the latest and greatest).  And if DistroA says they really
>>> want to stick to busybox 1.17.4 for a while, we should let that happen
>>> too.  But I don't think we want to have to carry on the recipes that
>>> angstrom-2008.1 wants and angstrom-2010.x wants and angstrom-2011.x
>>> wants and angstrom-2012.x want into 2013, in master.
>>
>> And noone says you should. At some point 2010.x works well enough to
>> force 2008.1 into hiding and start 201Y.x. The current situation where
>> the "unstable" 2010.x ended up in a product is largely due to the gcc
>> people breaking the NEON intrinsics interface API in between 4.3 and 4.5.
> 
> ick, I didn't know about that...

If anyone asks why uhd doesn't build for angstrom 2008, but it does
build for 2010, you now know why...

>>> For example, at some point we want to switch to libtool 2.4 only.  And
>>> that would certainly be a headache for angstrom-2008.1 (but we're glad,
>>> really! for angstrom-2010.x using 2.4 and testing and fixing things). So
>>> wouldn't it be a good thing to be able to say that if you want
>>> angstrom-2008.1 you do ... this ... and get the layers that give a good
>>> stable 2008.1, based on whatever policy Angstrom wants for doing
>>> updates?
>>
>> In the past the angstrom people created a stable branch and supported
>> that for a given release. The same can be done in the layering script,
>> where it would just lock down to certain revisions of various layers.
> 
> So, I think we agree.  Distros should be saying "if you want our stable
> release you should be over here..." and if you want our development WIP,
> you should be over here.

Exactly. Although it would make sense to have them coexist in the same
tree for a while while sorting out infrastructure. There are only so
many work hours in a day :)

>> But in the end if boils down to "Does OE wants to make life hard for
>> DISTROs or easy". Frans is firmly in the "make it hard" camp, I hope
>> others have a saner point of view.
>>
>> If you're forcing 90% of your users to put e.g. udev_162.bb in their
>> layer you're doing it wrong. But you're also doing it wrong if you have
>> 20 udev recipes :)
> 
> I think we also agree here.  But what's the rule of thumb(s) we want to
> have, to provide enough choice without too much headache?  As I said
> elsewhere, .inc files should probably be used a whole lot more, to help
> with the problem of recipe bugfixing and N recipes for an app with the
> problem.  We should probably also say that in addition to the "keep the
> last GPLv2+ version around" rule of thumb we should also have a "keep
> the latest stable release" around too.  But what else?  To use busybox
> as an example, do we really need to keep 1.18.0 and 1.18.1 around when
> we have 1.18.2?  How about if we make the delta between the 3 be just
> the SRC_URI + checksums?

Something like:

* keep gplv2 around if upstream switched to gplv3 at some point
* allow old stable, stable and dev (e.g. glib 2.24, 2.26 and 2.27+git)
* allow active contributers some leeway to test multiple subreleases
(e.g. busybox 1.18.[0-99]
* The maintainer can have as many versions as he wants to.
* toolchain is exempt from all that since having 20 binutils versions is
sometimes needed when building for 20 different archs[1].

But I seriously think we are overengineering this. We should have people
actually working on oe-core and meta-oe reach a consensus when
encountering problems.

regards,

Koen

[1] Although you can cheat by having different SRCREVs for each arch in
binutils_git.bb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFNNg6aMkyGM64RGpERAjdvAJoDSyv62VkZhr1Gs5pdWIYd0BZOdwCfYAjG
ManP61a+tHnNNuaC3a1kMWU=
=Wlmx
-----END PGP SIGNATURE-----




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 20:48           ` Tom Rini
  2011-01-18 22:05             ` Koen Kooi
@ 2011-01-18 22:48             ` Khem Raj
  2011-01-19  8:46               ` Frans Meulenbroeks
  1 sibling, 1 reply; 46+ messages in thread
From: Khem Raj @ 2011-01-18 22:48 UTC (permalink / raw)
  To: openembedded-devel

On Tue, Jan 18, 2011 at 12:48 PM, Tom Rini <tom_rini@mentor.com> wrote:
> On 01/18/2011 01:12 PM, Koen Kooi wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 18-01-11 17:54, Tom Rini wrote:
>>>
>>> On 01/18/2011 01:05 AM, Otavio Salvador wrote:
>>>>
>>>> On Tue, Jan 18, 2011 at 05:21, Graeme Gregory<dp@xora.org.uk>   wrote:
>>>>>
>>>>> On 17/01/2011 19:01, Frans Meulenbroeks wrote:
>>>>>>
>>>>>> - where possible stick to one recipe per package. This reduces the
>>>>>> maintenance work and reduces the QA nightmare of lots of different
>>>>>> permutations.
>>>>>> I feel one recipe per package should be the common case for
>>>>>> applications, and preferably also for libs (although I am well aware
>>>>>> that especially in the latter case multiple versions cannot always be
>>>>>> avoided).
>>>>>
>>>>> OE is not a distro so this is a non starter already, please don't bog
>>>>> down this discussion by re-opening this again. Angstrom 2008, Angstrom
>>>>> 2010, kaelios and slugos are all released distributions with different
>>>>> versions of apps just as a starter and they arent even near the total
>>>>> number of distros in OE.
>>>>
>>>> I disagree. I think having too many versions of a package just makes
>>>> difficult to get things done:
>>>>
>>>>   - it increases the amount of maintainence work;
>>>>   - has a bigger time to get bugs spoted;
>>>>
>>>> Users of old distros ought to use a specific repository and branch.
>>>> Master ought to be kept clean for 'next distro release'.
>>>
>>> I agree, at least going forward.  We must make it easier for
>>> distributions to say "here is my 'stable' release" and "here is my
>>> development release".
>>>
>>> First, I'm not picking on Angstrom here, really, I swear.  It's just a
>>> good example.
>>>
>>> But we also don't want to be unreasonable or unbending here.  We'll have
>>> to have multiple udevs (due to having different kernel versions as some
>>> HW isn't on the latest and greatest).  And if DistroA says they really
>>> want to stick to busybox 1.17.4 for a while, we should let that happen
>>> too.  But I don't think we want to have to carry on the recipes that
>>> angstrom-2008.1 wants and angstrom-2010.x wants and angstrom-2011.x
>>> wants and angstrom-2012.x want into 2013, in master.
>>
>> And noone says you should. At some point 2010.x works well enough to
>> force 2008.1 into hiding and start 201Y.x. The current situation where
>> the "unstable" 2010.x ended up in a product is largely due to the gcc
>> people breaking the NEON intrinsics interface API in between 4.3 and 4.5.
>
> ick, I didn't know about that...
>
>>> For example, at some point we want to switch to libtool 2.4 only.  And
>>> that would certainly be a headache for angstrom-2008.1 (but we're glad,
>>> really! for angstrom-2010.x using 2.4 and testing and fixing things). So
>>> wouldn't it be a good thing to be able to say that if you want
>>> angstrom-2008.1 you do ... this ... and get the layers that give a good
>>> stable 2008.1, based on whatever policy Angstrom wants for doing updates?
>>
>> In the past the angstrom people created a stable branch and supported
>> that for a given release. The same can be done in the layering script,
>> where it would just lock down to certain revisions of various layers.
>
> So, I think we agree.  Distros should be saying "if you want our stable
> release you should be over here..." and if you want our development WIP, you
> should be over here.
>
>> But in the end if boils down to "Does OE wants to make life hard for
>> DISTROs or easy". Frans is firmly in the "make it hard" camp, I hope
>> others have a saner point of view.
>>
>> If you're forcing 90% of your users to put e.g. udev_162.bb in their
>> layer you're doing it wrong. But you're also doing it wrong if you have
>> 20 udev recipes :)
>
> I think we also agree here.  But what's the rule of thumb(s) we want to
> have, to provide enough choice without too much headache?  As I said
> elsewhere, .inc files should probably be used a whole lot more, to help with
> the problem of recipe bugfixing and N recipes for an app with the problem.
>  We should probably also say that in addition to the "keep the last GPLv2+
> version around" rule of thumb we should also have a "keep the latest stable
> release" around too.  But what else?  To use busybox as an example, do we
> really need to keep 1.18.0 and 1.18.1 around when we have 1.18.2?  How about
> if we make the delta between the 3 be just the SRC_URI + checksums?


Well what to keep around can not be generalized so much. It also
depends upon the nature of releases for various packages
some packages have a major release and then push out bug fix releases
like busy box case you sited but there are packages
which only do major releases which can cause compatibility hassles as
new interfaces come and old ones are removed etc.
so I think it has to be flexible and mostly left to package
maintainers discretion as they know the nature of releases most but
I like the idea of keeping one GPLv2 release around and 1 latest
release around. If a distro pinned a version then that should
be considered as well. It is bad to sweep underneath a distros
pinnings. Then they have to rebuild and they get problems
as they have to make sure that if a package bump happens then they
should be able to define an upgrade path

Sometimes newer is not always better in case of udev the size has
increased over the time and some distro's may not like
it and there can be many such scenarios.

>
> --
> Tom Rini
> Mentor Graphics Corporation
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 20:12         ` Koen Kooi
  2011-01-18 20:48           ` Tom Rini
@ 2011-01-19  7:47           ` Frans Meulenbroeks
  2011-01-19  8:16             ` Martin Jansa
  1 sibling, 1 reply; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19  7:47 UTC (permalink / raw)
  To: openembedded-devel

2011/1/18 Koen Kooi <k.kooi@student.utwente.nl>:
> But in the end if boils down to "Does OE wants to make life hard for
> DISTROs or easy". Frans is firmly in the "make it hard" camp, I hope
> others have a saner point of view.

I don't want to make it hard for the distro's but I feel the distro's
should not make it hard for the package maintainers (which happens if
every distro pins its favourite version of recipe foo).
You're just pushing the burden of the distro choices onto the package
maintainer.
(and frankly, I feel the package maintainer has a much harder job than
the distro maintainers)

But then again you already clearly exhibited on several occasions in
the past how much you care about other users and distros. :-(

Root cause of your problem is that angstrom wants to provide a live
binary feed of git head. Something that is bound to encounter problems
on occasions. Git head it bleeding edge, so sometimes it bleeds. That
is an effect of moving forward. The alternative is reduce pace and
loose momentum (which is what you are doing right now).
And anyone with some sense of quality can explain you that publishing
untested binary packages is not really a good idea quality wise.

I think the angstrom users would be better off with releases.
>
> If you're forcing 90% of your users to put e.g. udev_162.bb in their
> layer you're doing it wrong. But you're also doing it wrong if you have
> 20 udev recipes :)

We fully agree on both cases. Then again hardly anyone seems to be
willing to clean up, and those who do only face headwind.
And frankly speaking a good package maintainer would have prevented
those 20 (well actually about 10) recipes to happen, keeping just one
or a few versions (that are kept for good reasons).

Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  7:47           ` Frans Meulenbroeks
@ 2011-01-19  8:16             ` Martin Jansa
  0 siblings, 0 replies; 46+ messages in thread
From: Martin Jansa @ 2011-01-19  8:16 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1453 bytes --]

On Wed, Jan 19, 2011 at 08:47:42AM +0100, Frans Meulenbroeks wrote:
> 2011/1/18 Koen Kooi <k.kooi@student.utwente.nl>:
> > But in the end if boils down to "Does OE wants to make life hard for
> > DISTROs or easy". Frans is firmly in the "make it hard" camp, I hope
> > others have a saner point of view.
> 
> I don't want to make it hard for the distro's but I feel the distro's
> should not make it hard for the package maintainers (which happens if
> every distro pins its favourite version of recipe foo).
> You're just pushing the burden of the distro choices onto the package
> maintainer.

I don't see any problem with active, well-maintained distro (like
Ångström is) pinning different versions. Everytime I see package
maintainer asking for pinned version change (like Eric does for
busybox), he receives Ack from distro maintainer with reasonable RTT,
or receives the reason why that distro needs this specific version for 
longer and they can discuss if it will be moved to distro specific layer 
or stays in core when it could be usefull for other distros as well.

Problem with pinned versions in OE I had only with stalled distro.confs
which weren't changed in years, nobody knows if there is someone still
building it etc.. But as Graeme said, those unmaintained distros won't
go to openembedded-core without someone actually moving them :).

Regards,
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 22:05             ` Koen Kooi
@ 2011-01-19  8:45               ` Frans Meulenbroeks
  2011-01-19  8:58                 ` Graeme Gregory
  2011-01-19  9:25                 ` Frans Meulenbroeks
  0 siblings, 2 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19  8:45 UTC (permalink / raw)
  To: openembedded-devel

2011/1/18 Koen Kooi <k.kooi@student.utwente.nl>:
 if we make the delta between the 3 be just
>> the SRC_URI + checksums?
>
> Something like:
>
> * keep gplv2 around if upstream switched to gplv3 at some point

agree; as said before suggest to reflect the switch in the name of the recipe.

> * allow old stable, stable and dev (e.g. glib 2.24, 2.26 and 2.27+git)

I'd say retire old stable after a while (e.g. after stable is there
for half a year, unless of course there are compelling reasons to keep
it).
Wrt dev: if we want to go to a maintainers model similar to u-boot and
linux, I would expect this to live in a separate branch or layer.
Once proven stable the maintainers pulls it into the core archive.

> * allow active contributers some leeway to test multiple subreleases
> (e.g. busybox 1.18.[0-99]

Again here, I feel it is much better to put it into a different branch
or layer, and once stable it is pulled into the mainline.
Again similar to what Linus and Wolfgang do.

> * The maintainer can have as many versions as he wants to.

my preference: not in the main layer.

> * toolchain is exempt from all that since having 20 binutils versions is
> sometimes needed when building for 20 different archs[1].

Fully agree.
>
> But I seriously think we are overengineering this. We should have people
> actually working on oe-core and meta-oe reach a consensus when
> encountering problems.

I beg to disagree on this.
We have a unique opportunity at hand to improve, so it seems wise to
raise the bar quality wise.

E.g. wrt what Richard wrote in the start post

[quote]
* Creation of a subset of metadata that has a consistent high quality
 standard:
   - removal of legacy components (pkgconfig hacks, libtool hacks,
     legacy staging, unneeded compiler arguments)
   - consistent metadata layout, spacing, use of variables
   - migration away from outdated practises (e.g. use BBCLASSEXTEND)
[/quote]

I feel when migrating recipes we should also assure that they adhere
to a certain quality standard. That would need us to define a minimal
standard though.
Currently I feel we are just moving recipes, only fixing what is
really needed to get things running (like removing legacy staging)
whereas we could do much better.
There are currently some things in OE that could use some improvement
and we should take the opportunity to improve.l

And yes, I have no problem in spending time to improve things, but if
there is no consensus on where we should go this is not going to be
effective (and actually makes it harder to improve quality wise).
To take the bikeshed analogy up again. It does not really help if
everyone start to paint in his own favourite color (unless maybe if
you are still stuck in the flower power era).

Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-18 22:48             ` Khem Raj
@ 2011-01-19  8:46               ` Frans Meulenbroeks
  2011-01-19  8:52                 ` Graeme Gregory
                                   ` (2 more replies)
  0 siblings, 3 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19  8:46 UTC (permalink / raw)
  To: openembedded-devel

2011/1/18 Khem Raj <raj.khem@gmail.com>:
> On Tue, Jan 18, 2011 at 12:48 PM, Tom Rini <tom_rini@mentor.com> wrote:
>> I think we also agree here.  But what's the rule of thumb(s) we want to
>> have, to provide enough choice without too much headache?  As I said
>> elsewhere, .inc files should probably be used a whole lot more, to help with
>> the problem of recipe bugfixing and N recipes for an app with the problem.
>>  We should probably also say that in addition to the "keep the last GPLv2+
>> version around" rule of thumb we should also have a "keep the latest stable
>> release" around too.  But what else?  To use busybox as an example, do we
>> really need to keep 1.18.0 and 1.18.1 around when we have 1.18.2?  How about
>> if we make the delta between the 3 be just the SRC_URI + checksums?
>
>
> Well what to keep around can not be generalized so much. It also
> depends upon the nature of releases for various packages
> some packages have a major release and then push out bug fix releases
> like busy box case you sited but there are packages
> which only do major releases which can cause compatibility hassles as
> new interfaces come and old ones are removed etc.
> so I think it has to be flexible and mostly left to package
> maintainers discretion as they know the nature of releases most but
> I like the idea of keeping one GPLv2 release around and 1 latest
> release around. If a distro pinned a version then that should
> be considered as well. It is bad to sweep underneath a distros
> pinnings. Then they have to rebuild and they get problems
> as they have to make sure that if a package bump happens then they
> should be able to define an upgrade path
>
> Sometimes newer is not always better in case of udev the size has
> increased over the time and some distro's may not like
> it and there can be many such scenarios.
>

I wholehearthy agree with the proposal that it is left to the package
maintainers discretion.
Wrt the GPLv2+ version. I suggest to reflect this in the name.
E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
samba-gplv3). Then it immediately becomes obvious why there are two
versions.
Similarly with versions that became a lot fatter over time (which imho
is a good reason to keep the old version).

Frans.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:46               ` Frans Meulenbroeks
@ 2011-01-19  8:52                 ` Graeme Gregory
  2011-01-19  9:12                   ` Frans Meulenbroeks
  2011-01-19 16:56                 ` Khem Raj
  2011-01-22 18:20                 ` Tom Rini
  2 siblings, 1 reply; 46+ messages in thread
From: Graeme Gregory @ 2011-01-19  8:52 UTC (permalink / raw)
  To: openembedded-devel

On 19/01/2011 08:46, Frans Meulenbroeks wrote:
> 2011/1/18 Khem Raj <raj.khem@gmail.com>:
>> On Tue, Jan 18, 2011 at 12:48 PM, Tom Rini <tom_rini@mentor.com> wrote:
>>> I think we also agree here.  But what's the rule of thumb(s) we want to
>>> have, to provide enough choice without too much headache?  As I said
>>> elsewhere, .inc files should probably be used a whole lot more, to help with
>>> the problem of recipe bugfixing and N recipes for an app with the problem.
>>>  We should probably also say that in addition to the "keep the last GPLv2+
>>> version around" rule of thumb we should also have a "keep the latest stable
>>> release" around too.  But what else?  To use busybox as an example, do we
>>> really need to keep 1.18.0 and 1.18.1 around when we have 1.18.2?  How about
>>> if we make the delta between the 3 be just the SRC_URI + checksums?
>>
>> Well what to keep around can not be generalized so much. It also
>> depends upon the nature of releases for various packages
>> some packages have a major release and then push out bug fix releases
>> like busy box case you sited but there are packages
>> which only do major releases which can cause compatibility hassles as
>> new interfaces come and old ones are removed etc.
>> so I think it has to be flexible and mostly left to package
>> maintainers discretion as they know the nature of releases most but
>> I like the idea of keeping one GPLv2 release around and 1 latest
>> release around. If a distro pinned a version then that should
>> be considered as well. It is bad to sweep underneath a distros
>> pinnings. Then they have to rebuild and they get problems
>> as they have to make sure that if a package bump happens then they
>> should be able to define an upgrade path
>>
>> Sometimes newer is not always better in case of udev the size has
>> increased over the time and some distro's may not like
>> it and there can be many such scenarios.
>>
> I wholehearthy agree with the proposal that it is left to the package
> maintainers discretion.
> Wrt the GPLv2+ version. I suggest to reflect this in the name.
> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
> samba-gplv3). Then it immediately becomes obvious why there are two
> versions.
> Similarly with versions that became a lot fatter over time (which imho
> is a good reason to keep the old version).
I am totally against this idea, it just makes a total mess of the
namespace. We have the ability to put comments in bitbake files use it.

Graeme




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:45               ` Frans Meulenbroeks
@ 2011-01-19  8:58                 ` Graeme Gregory
  2011-01-19  9:16                   ` Frans Meulenbroeks
  2011-01-19  9:25                 ` Frans Meulenbroeks
  1 sibling, 1 reply; 46+ messages in thread
From: Graeme Gregory @ 2011-01-19  8:58 UTC (permalink / raw)
  To: openembedded-devel

On 19/01/2011 08:45, Frans Meulenbroeks wrote:
>
>> * The maintainer can have as many versions as he wants to.
> my preference: not in the main layer.
>

I suspect all that is going to happen with this attitude is the recipes
in the main layer will end up not maintained as the recipe maintainer
will just maintain them in his own layer. Leading to fracturing of OE
which is what we are trying to prevent. Sometimes you are just going to
have to accept that the recipe maintainer knows best and knows what he
is doing.

Graeme




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:52                 ` Graeme Gregory
@ 2011-01-19  9:12                   ` Frans Meulenbroeks
  2011-01-19  9:19                     ` Graeme Gregory
  2011-01-19 11:31                     ` Joshua Lock
  0 siblings, 2 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19  9:12 UTC (permalink / raw)
  To: openembedded-devel

2011/1/19 Graeme Gregory <dp@xora.org.uk>:

>> I wholehearthy agree with the proposal that it is left to the package
>> maintainers discretion.
>> Wrt the GPLv2+ version. I suggest to reflect this in the name.
>> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
>> samba-gplv3). Then it immediately becomes obvious why there are two
>> versions.
>> Similarly with versions that became a lot fatter over time (which imho
>> is a good reason to keep the old version).
> I am totally against this idea, it just makes a total mess of the
> namespace. We have the ability to put comments in bitbake files use it.

I don't think there are that many recipes for which this is relevant,
so the namespace clutter is limited.
Comments in bb files may work equally well. Problem is that those
comments are not written in the files.

E.g. you mentioned you wanted to keep an older abiword version because
it is much smaller. Fine, but I have no idea which of the 5 versions
you want to keep.
(btw, talking about namespace clutter: there is also
abiword-embedded_2.6.8.bb) and 2.8 recipes including a 2.5 inc seems
also an issue that might be worthwile fixing)

Wrt the message of Martin: I did a quick peek,
include/angstrom-2008-preferred-versions.inc pins about 60 recipes
(actually a litlte bit less if you count libtool libtool-native
libtool-cross libtool-sdk etc as one). Most of them are for libs. This
also seems to hold for other distro's.

Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:58                 ` Graeme Gregory
@ 2011-01-19  9:16                   ` Frans Meulenbroeks
  0 siblings, 0 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19  9:16 UTC (permalink / raw)
  To: openembedded-devel

2011/1/19 Graeme Gregory <dp@xora.org.uk>:
> On 19/01/2011 08:45, Frans Meulenbroeks wrote:
>>
>>> * The maintainer can have as many versions as he wants to.
>> my preference: not in the main layer.
>>
>
> I suspect all that is going to happen with this attitude is the recipes
> in the main layer will end up not maintained as the recipe maintainer
> will just maintain them in his own layer. Leading to fracturing of OE
> which is what we are trying to prevent. Sometimes you are just going to
> have to accept that the recipe maintainer knows best and knows what he
> is doing.

Agree.
Unfortunately some of the root causes of the problems we are having in
the current tree are that for lots of recipes the maintainer is not
known or gone or does not do a very good job.
In case of a good recipe maintainer (like e.g. khem with gcc) this is
not a problem. Unfortunately not all our recipes are as
well-maintained as gcc.

Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  9:12                   ` Frans Meulenbroeks
@ 2011-01-19  9:19                     ` Graeme Gregory
  2011-01-19 11:31                     ` Joshua Lock
  1 sibling, 0 replies; 46+ messages in thread
From: Graeme Gregory @ 2011-01-19  9:19 UTC (permalink / raw)
  To: openembedded-devel

On 19/01/2011 09:12, Frans Meulenbroeks wrote:
>
> E.g. you mentioned you wanted to keep an older abiword version because
> it is much smaller. Fine, but I have no idea which of the 5 versions
> you want to keep.
> (btw, talking about namespace clutter: there is also
> abiword-embedded_2.6.8.bb) and 2.8 recipes including a 2.5 inc seems
> also an issue that might be worthwile fixing)
>
abiword embedded is a different product with a different UI.

Graeme




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:45               ` Frans Meulenbroeks
  2011-01-19  8:58                 ` Graeme Gregory
@ 2011-01-19  9:25                 ` Frans Meulenbroeks
  2011-01-22 18:16                   ` Tom Rini
  1 sibling, 1 reply; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19  9:25 UTC (permalink / raw)
  To: openembedded-devel

2011/1/19 Frans Meulenbroeks <fransmeulenbroeks@gmail.com>:
> 2011/1/18 Koen Kooi <k.kooi@student.utwente.nl>:
>  if we make the delta between the 3 be just
>>> the SRC_URI + checksums?
>>
>> Something like:
>>
>> * keep gplv2 around if upstream switched to gplv3 at some point
>
> agree; as said before suggest to reflect the switch in the name of the recipe.
>
>> * allow old stable, stable and dev (e.g. glib 2.24, 2.26 and 2.27+git)
>
> I'd say retire old stable after a while (e.g. after stable is there
> for half a year, unless of course there are compelling reasons to keep
> it).
> Wrt dev: if we want to go to a maintainers model similar to u-boot and
> linux, I would expect this to live in a separate branch or layer.
> Once proven stable the maintainers pulls it into the core archive.
>
>> * allow active contributers some leeway to test multiple subreleases
>> (e.g. busybox 1.18.[0-99]
>
> Again here, I feel it is much better to put it into a different branch
> or layer, and once stable it is pulled into the mainline.
> Again similar to what Linus and Wolfgang do.
>
>> * The maintainer can have as many versions as he wants to.
>
> my preference: not in the main layer.
>
>> * toolchain is exempt from all that since having 20 binutils versions is
>> sometimes needed when building for 20 different archs[1].
>
> Fully agree.
>>
>> But I seriously think we are overengineering this. We should have people
>> actually working on oe-core and meta-oe reach a consensus when
>> encountering problems.
>
> I beg to disagree on this.
> We have a unique opportunity at hand to improve, so it seems wise to
> raise the bar quality wise.
>
> E.g. wrt what Richard wrote in the start post
>
> [quote]
> * Creation of a subset of metadata that has a consistent high quality
>  standard:
>   - removal of legacy components (pkgconfig hacks, libtool hacks,
>     legacy staging, unneeded compiler arguments)
>   - consistent metadata layout, spacing, use of variables
>   - migration away from outdated practises (e.g. use BBCLASSEXTEND)
> [/quote]
>
> I feel when migrating recipes we should also assure that they adhere
> to a certain quality standard. That would need us to define a minimal
> standard though.
> Currently I feel we are just moving recipes, only fixing what is
> really needed to get things running (like removing legacy staging)
> whereas we could do much better.
> There are currently some things in OE that could use some improvement
> and we should take the opportunity to improve.l
>
> And yes, I have no problem in spending time to improve things, but if
> there is no consensus on where we should go this is not going to be
> effective (and actually makes it harder to improve quality wise).
> To take the bikeshed analogy up again. It does not really help if
> everyone start to paint in his own favourite color (unless maybe if
> you are still stuck in the flower power era).
>
> Frans
>
I know it is fairly schizofrenic to reply on ones own post, but I just
peeked into the meta-oe layer and found an excellent case on what we
should try to avoid when bringing over recipes from oe to meta-oe.
path: root/recipes-graphics/directfb
Mode	Name	Size	
d---------	++dfb	48	logplain
-rw-r--r--	++dfb_1.0.0.bb	588	logplain
d---------	directfb-1.2.8	50	logplain
d---------	directfb-1.4.6	41	logplain
-rw-r--r--	directfb-examples_1.0.0.bb	406	logplain
-rw-r--r--	directfb-examples_1.2.0.bb	409	logplain
-rw-r--r--	directfb.inc	2038	logplain
-rw-r--r--	directfb_1.4.6.bb	615	logplain
d---------	files	415	logplain
d---------	fusionsound-1.1.0+git20070709	54	logplain
-rw-r--r--	fusionsound_1.1.0+git20070709.bb	1479	logplain
-rw-r--r--	lite_0.9.26+cvs20070207.bb	1140	logplain

Why is there a directfb-examples_1.0.0.bb? In OE no-one is pinning
this version, and given the fact that these are examples there is
probably little reason to keep it.
Something similar for directfb-1.2.8. Maybe I overlooked things but I
could not find a the user of this dir. I guess it could be gone.

If we want to improve on quality it is probably not a bad idea to at
least have a checklist with improvement actions that should be done
when migrating recipes (e.g. remove stale patches, unused & unpinned
older versions etc etc). I know it takes time, but it can also
increase the overall quality. Let's take that opportunity.

Frans.

PS: as part of the migration it might also be a good plan to see if
recipes should be updated. I don't know too much about directfb but
maybe it makes sense to move to newer versions of e.g. fusionsound and
lite.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  9:12                   ` Frans Meulenbroeks
  2011-01-19  9:19                     ` Graeme Gregory
@ 2011-01-19 11:31                     ` Joshua Lock
  2011-01-19 12:44                       ` Frans Meulenbroeks
  1 sibling, 1 reply; 46+ messages in thread
From: Joshua Lock @ 2011-01-19 11:31 UTC (permalink / raw)
  To: openembedded-devel

On Wed, 2011-01-19 at 10:12 +0100, Frans Meulenbroeks wrote:
> 2011/1/19 Graeme Gregory <dp@xora.org.uk>:
> 
> >> I wholehearthy agree with the proposal that it is left to the package
> >> maintainers discretion.
> >> Wrt the GPLv2+ version. I suggest to reflect this in the name.
> >> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
> >> samba-gplv3). Then it immediately becomes obvious why there are two
> >> versions.
> >> Similarly with versions that became a lot fatter over time (which imho
> >> is a good reason to keep the old version).
> > I am totally against this idea, it just makes a total mess of the
> > namespace. We have the ability to put comments in bitbake files use it.
> 
> I don't think there are that many recipes for which this is relevant,
> so the namespace clutter is limited.
> Comments in bb files may work equally well. Problem is that those
> comments are not written in the files.

Surely that what the LICENSE field in the metadata is for?

Aside: In Poky we have various features to ensure recipes have license
information included, and functionality to blacklist packages of a
certain license - these will doubtless be merged into oe-core too.

Cheers,
Joshua
-- 
Joshua Lock
        Intel Open Source Technology Centre




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 11:31                     ` Joshua Lock
@ 2011-01-19 12:44                       ` Frans Meulenbroeks
  2011-01-19 15:09                         ` Mike Westerhof
  0 siblings, 1 reply; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19 12:44 UTC (permalink / raw)
  To: openembedded-devel

2011/1/19 Joshua Lock <josh@linux.intel.com>:
> On Wed, 2011-01-19 at 10:12 +0100, Frans Meulenbroeks wrote:
>> 2011/1/19 Graeme Gregory <dp@xora.org.uk>:
>>
>> >> I wholehearthy agree with the proposal that it is left to the package
>> >> maintainers discretion.
>> >> Wrt the GPLv2+ version. I suggest to reflect this in the name.
>> >> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
>> >> samba-gplv3). Then it immediately becomes obvious why there are two
>> >> versions.
>> >> Similarly with versions that became a lot fatter over time (which imho
>> >> is a good reason to keep the old version).
>> > I am totally against this idea, it just makes a total mess of the
>> > namespace. We have the ability to put comments in bitbake files use it.
>>
>> I don't think there are that many recipes for which this is relevant,
>> so the namespace clutter is limited.
>> Comments in bb files may work equally well. Problem is that those
>> comments are not written in the files.
>
> Surely that what the LICENSE field in the metadata is for?

That works for gpl v2 vs v3, but if there are other reasons to keep an
old recipe (e.g. footprint) it would also be nice to record it one way
or another.
>
> Aside: In Poky we have various features to ensure recipes have license
> information included, and functionality to blacklist packages of a
> certain license - these will doubtless be merged into oe-core too.

I assume so

Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 12:44                       ` Frans Meulenbroeks
@ 2011-01-19 15:09                         ` Mike Westerhof
  2011-01-19 15:44                           ` Frans Meulenbroeks
  2011-01-19 18:03                           ` Khem Raj
  0 siblings, 2 replies; 46+ messages in thread
From: Mike Westerhof @ 2011-01-19 15:09 UTC (permalink / raw)
  To: openembedded-devel

As I read this thread, I find that as a distro maintainer, I'm a bit
unclear on a few points.  Pardon me if I've just failed to read/follow
this thread...

- So if I wish (or am forced) to use "layers", where do they come from?
 Am I then responsible for finding a git hosting service, arranging for
CIA commit messages, etc?  Or will each layer be hosted alongside the
main OE repo?  Or have we not yet considered the added administrative
burden of multiple repos?

- Does this not "silo" things even more?  For example, I have found life
has gotten much easier now that other autobuilders are building SlugOS
-- but what incentive do people have if they have to deal with adding
layers and all that, especially if the "SlugOS layer" ends up with a
conflict in some fashion with the "Angstrom layer"?  And
correspondingly, if I am struggling with some issue, would it not be
less likely that I could get assistance and guidance from #OE if I'm
using some layer?

- And unless I'm not missing something with the layers proposal (and I
hope I am), is it not most likely that we'll end up with the following
scenario:

  a) Larger distros eventually fork OE-core and go off on their own as
they find that their "layer" becomes more and more substantial compared
to OE-core, especially when "people-originated conflicts" occur rather
than "technically-originated conflicts" (a single repo tends to force
resolution of the former).

  b) Small distros are forced to comply with nothing but OE-core, or
perish -- the cost of maintaining the extra infrastructure (git, cgit,
CIA, etc, etc) along with the added complexity of teaching new distro
developers about this new layer of stuff) is simply too high, not to
mention that creating a layer for distro-specific stuff might be an
instant killer in terms of autobuilder and assistance from the OE
community at large (I ask myself -- do *I* intend to pull the Angstrom
layer and build it?  I've done so in the past, but only because it was
trivial to kick off such a build... but with layers, it seems it will be
far harder to do so -- unless I'm missing something).

  c) Obscure/experimental distros simply die off, or find an alternative
to OE.  The learning curve is quite steep already, and without access to
shared knowledge, shared recipes, and the assistance of the community,
what is there to encourage a newcomer to use OE?


I'm putting on my asbestos skivvies, since I expect this must have
already been discussed :)

-Mike (mwester)



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 15:09                         ` Mike Westerhof
@ 2011-01-19 15:44                           ` Frans Meulenbroeks
  2011-01-19 18:03                           ` Khem Raj
  1 sibling, 0 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-19 15:44 UTC (permalink / raw)
  To: openembedded-devel

2011/1/19 Mike Westerhof <mike@mwester.net>:
> As I read this thread, I find that as a distro maintainer, I'm a bit
> unclear on a few points.  Pardon me if I've just failed to read/follow
> this thread...
>
> - So if I wish (or am forced) to use "layers", where do they come from?
>  Am I then responsible for finding a git hosting service, arranging for
> CIA commit messages, etc?  Or will each layer be hosted alongside the
> main OE repo?  Or have we not yet considered the added administrative
> burden of multiple repos?

I hope the latter, but this is not fully ironed out.

>
> - Does this not "silo" things even more?  For example, I have found life
> has gotten much easier now that other autobuilders are building SlugOS

You're welcome! Glad you like it (that reminds me that I still have to
add last weeks result to the table)

> -- but what incentive do people have if they have to deal with adding
> layers and all that, especially if the "SlugOS layer" ends up with a
> conflict in some fashion with the "Angstrom layer"?  And
> correspondingly, if I am struggling with some issue, would it not be
> less likely that I could get assistance and guidance from #OE if I'm
> using some layer?

My impression would be that the slugos layer would only contain the
slugos distro specific stuff.
E.g. things like conf/distro/slugos.conf and what is included by it,
and slugos specific recipes (probably things like upslug2, although
technically they are nslu2 specific not slugos specific, so they could
also go into an nslu2 machine layer).
But this has not really been discussed as far as i recall.
>
> - And unless I'm not missing something with the layers proposal (and I
> hope I am), is it not most likely that we'll end up with the following
> scenario:
>
>  a) Larger distros eventually fork OE-core and go off on their own as
> they find that their "layer" becomes more and more substantial compared
> to OE-core, especially when "people-originated conflicts" occur rather
> than "technically-originated conflicts" (a single repo tends to force
> resolution of the former).

I hope not. Actually as OE-core has also the yocto momentum behind it
I guess that forks will probably die because the additional effort is
not available or does not pay off well.
Preferably we should keep it one repo.
Having a good maintainer/integrator also helps (with authority and
good judgement).
The process for OE-core would probably be similar to what u-boot and
the kernel do.
>
>  b) Small distros are forced to comply with nothing but OE-core, or
> perish -- the cost of maintaining the extra infrastructure (git, cgit,
> CIA, etc, etc) along with the added complexity of teaching new distro
> developers about this new layer of stuff) is simply too high, not to
> mention that creating a layer for distro-specific stuff might be an
> instant killer in terms of autobuilder and assistance from the OE
> community at large (I ask myself -- do *I* intend to pull the Angstrom
> layer and build it?  I've done so in the past, but only because it was
> trivial to kick off such a build... but with layers, it seems it will be
> far harder to do so -- unless I'm missing something).

This is definitely  an issue we need to address.
>
>  c) Obscure/experimental distros simply die off, or find an alternative
> to OE.  The learning curve is quite steep already, and without access to
> shared knowledge, shared recipes, and the assistance of the community,
> what is there to encourage a newcomer to use OE?

I'd hope there is lots of sharing and little effort needed to create a
new layer.

>
>
> I'm putting on my asbestos skivvies, since I expect this must have
> already been discussed :)

:-)

Enjoy, Frans.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:46               ` Frans Meulenbroeks
  2011-01-19  8:52                 ` Graeme Gregory
@ 2011-01-19 16:56                 ` Khem Raj
  2011-01-22 18:20                 ` Tom Rini
  2 siblings, 0 replies; 46+ messages in thread
From: Khem Raj @ 2011-01-19 16:56 UTC (permalink / raw)
  To: openembedded-devel

On (19/01/11 09:46), Frans Meulenbroeks wrote:
> 2011/1/18 Khem Raj <raj.khem@gmail.com>:
> > On Tue, Jan 18, 2011 at 12:48 PM, Tom Rini <tom_rini@mentor.com> wrote:
> >> I think we also agree here.  But what's the rule of thumb(s) we want to
> >> have, to provide enough choice without too much headache?  As I said
> >> elsewhere, .inc files should probably be used a whole lot more, to help with
> >> the problem of recipe bugfixing and N recipes for an app with the problem.
> >>  We should probably also say that in addition to the "keep the last GPLv2+
> >> version around" rule of thumb we should also have a "keep the latest stable
> >> release" around too.  But what else?  To use busybox as an example, do we
> >> really need to keep 1.18.0 and 1.18.1 around when we have 1.18.2?  How about
> >> if we make the delta between the 3 be just the SRC_URI + checksums?
> >
> >
> > Well what to keep around can not be generalized so much. It also
> > depends upon the nature of releases for various packages
> > some packages have a major release and then push out bug fix releases
> > like busy box case you sited but there are packages
> > which only do major releases which can cause compatibility hassles as
> > new interfaces come and old ones are removed etc.
> > so I think it has to be flexible and mostly left to package
> > maintainers discretion as they know the nature of releases most but
> > I like the idea of keeping one GPLv2 release around and 1 latest
> > release around. If a distro pinned a version then that should
> > be considered as well. It is bad to sweep underneath a distros
> > pinnings. Then they have to rebuild and they get problems
> > as they have to make sure that if a package bump happens then they
> > should be able to define an upgrade path
> >
> > Sometimes newer is not always better in case of udev the size has
> > increased over the time and some distro's may not like
> > it and there can be many such scenarios.
> >
> 
> I wholehearthy agree with the proposal that it is left to the package
> maintainers discretion.
> Wrt the GPLv2+ version. I suggest to reflect this in the name.
> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
> samba-gplv3). Then it immediately becomes obvious why there are two
> versions.
nope, LICENSE field should denote that. Renaming recipes this way does
not look nice.

> Similarly with versions that became a lot fatter over time (which imho
> is a good reason to keep the old version).
> 
> Frans.
> 
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 15:09                         ` Mike Westerhof
  2011-01-19 15:44                           ` Frans Meulenbroeks
@ 2011-01-19 18:03                           ` Khem Raj
  2011-01-19 21:55                             ` C Michael Sundius
  1 sibling, 1 reply; 46+ messages in thread
From: Khem Raj @ 2011-01-19 18:03 UTC (permalink / raw)
  To: openembedded-devel

On Wed, Jan 19, 2011 at 7:09 AM, Mike Westerhof <mike@mwester.net> wrote:
> As I read this thread, I find that as a distro maintainer, I'm a bit
> unclear on a few points.  Pardon me if I've just failed to read/follow
> this thread...
>
> - So if I wish (or am forced) to use "layers", where do they come from?
>  Am I then responsible for finding a git hosting service, arranging for
> CIA commit messages, etc?  Or will each layer be hosted alongside the
> main OE repo?  Or have we not yet considered the added administrative
> burden of multiple repos?

well if distro is kept in sync with oe-core it should not need many bits.
may be oe-meta will hold all the remaining bits that it might need on top

>
> - Does this not "silo" things even more?  For example, I have found life
> has gotten much easier now that other autobuilders are building SlugOS
> -- but what incentive do people have if they have to deal with adding
> layers and all that, especially if the "SlugOS layer" ends up with a
> conflict in some fashion with the "Angstrom layer"?  And
> correspondingly, if I am struggling with some issue, would it not be
> less likely that I could get assistance and guidance from #OE if I'm
> using some layer?

yes we are learning about layers and certainly there will be issues
there is priority so if you want to override over angstrom-layer you can
and still use angstrom-layer. I think if we do a good job of oe-core then
the possibility of conflicts could be minimised.

>
> - And unless I'm not missing something with the layers proposal (and I
> hope I am), is it not most likely that we'll end up with the following
> scenario:
>
>  a) Larger distros eventually fork OE-core and go off on their own as
> they find that their "layer" becomes more and more substantial compared
> to OE-core, especially when "people-originated conflicts" occur rather
> than "technically-originated conflicts" (a single repo tends to force
> resolution of the former).

if we do a lousy work in maintaining oe-core chances of this happening are less
IOW we have to keep the distros in mind when we work on oe-core and cater
to their needs absolutely in a portable way

>
>  b) Small distros are forced to comply with nothing but OE-core, or
> perish -- the cost of maintaining the extra infrastructure (git, cgit,
> CIA, etc, etc) along with the added complexity of teaching new distro
> developers about this new layer of stuff) is simply too high, not to
> mention that creating a layer for distro-specific stuff might be an
> instant killer in terms of autobuilder and assistance from the OE
> community at large (I ask myself -- do *I* intend to pull the Angstrom
> layer and build it?  I've done so in the past, but only because it was
> trivial to kick off such a build... but with layers, it seems it will be
> far harder to do so -- unless I'm missing something).
>

The meta-oe layer will host I think all bits out of oe-core

>  c) Obscure/experimental distros simply die off, or find an alternative
> to OE.  The learning curve is quite steep already, and without access to
> shared knowledge, shared recipes, and the assistance of the community,
> what is there to encourage a newcomer to use OE?

well its just that you will have two or more repos to pull from to
make what you get in one shot
from OE as of today
>
>
> I'm putting on my asbestos skivvies, since I expect this must have
> already been discussed :)
>
> -Mike (mwester)
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 18:03                           ` Khem Raj
@ 2011-01-19 21:55                             ` C Michael Sundius
  2011-01-19 22:01                               ` C Michael Sundius
  2011-01-19 22:23                               ` Khem Raj
  0 siblings, 2 replies; 46+ messages in thread
From: C Michael Sundius @ 2011-01-19 21:55 UTC (permalink / raw)
  To: openembedded-devel

It seems to me that this is a bit of a battle between the package
maintainers and the distro maintainers.. Looking at this from my managements
side of things, we use OE as a tool and its really just a means to the end.
our customers demand that we do not change things (versions of software),
they demand stability and they view a change in busybox or anything else a
threat to stability. our management has also made an edict that we can not
use gplv3. For completely non technical reasons we simply cannot move to new
package versions without a substantial business justification. I suspect
that that there are many (more than you realize) folk out there who are
using OE for their own distro. If you simply whack package versions because
something newer came out you will have these people maintaining separate
recipes and we'll be swamped with the load and this tool will loose one of
its best attributes.

The comment that disturbed me was that distros should just move ahead
"because its making things hard for the package maintainer". That doesn't
wash with me because if people are using your package then you should
support it or let someone else be the maintainer. In essence the distro's
use of that package are your customers and the reason you have a job. OE
does not exist as a product, rather a tool that enables customers, you can't
create this in a vacuum without understanding who is using it.

distro maintainers are not all dumb and if they are they'll be the last
single one using an outdated version of the software. When that happens a
smart package maintainer will call it out leave out the old package.
Further, it would be nice for a warning to take place so that it might have
a "depracated" tag associated with the recipe for one release cycle to see
if anyone cribs.

So I'm standing with the guy w/ asbestos short on. I'd like to see that OE
err on the side of "do no harm" to existing users. Its hard enough to rally
the troops to move to updated packages much less updated meta without you
leaving perfectly reasonable versions of software out of oe-core.

mike


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 21:55                             ` C Michael Sundius
@ 2011-01-19 22:01                               ` C Michael Sundius
  2011-01-19 22:22                                 ` Philip Balister
  2011-01-19 22:23                               ` Khem Raj
  1 sibling, 1 reply; 46+ messages in thread
From: C Michael Sundius @ 2011-01-19 22:01 UTC (permalink / raw)
  To: openembedded-devel

in rereading this I don't want to seem ungrateful, since we've certainly
benefited from the great effort on everyones part (package and distro
maintainers, yacto and OE... everyone).

On Wed, Jan 19, 2011 at 1:55 PM, C Michael Sundius <msundius@sundius.com>wrote:

>
> It seems to me that this is a bit of a battle between the package
> maintainers and the distro maintainers.. Looking at this from my managements
> side of things, we use OE as a tool and its really just a means to the end.
> our customers demand that we do not change things (versions of software),
> they demand stability and they view a change in busybox or anything else a
> threat to stability. our management has also made an edict that we can not
> use gplv3. For completely non technical reasons we simply cannot move to new
> package versions without a substantial business justification. I suspect
> that that there are many (more than you realize) folk out there who are
> using OE for their own distro. If you simply whack package versions because
> something newer came out you will have these people maintaining separate
> recipes and we'll be swamped with the load and this tool will loose one of
> its best attributes.
>
> The comment that disturbed me was that distros should just move ahead
> "because its making things hard for the package maintainer". That doesn't
> wash with me because if people are using your package then you should
> support it or let someone else be the maintainer. In essence the distro's
> use of that package are your customers and the reason you have a job. OE
> does not exist as a product, rather a tool that enables customers, you can't
> create this in a vacuum without understanding who is using it.
>
> distro maintainers are not all dumb and if they are they'll be the last
> single one using an outdated version of the software. When that happens a
> smart package maintainer will call it out leave out the old package.
> Further, it would be nice for a warning to take place so that it might have
> a "depracated" tag associated with the recipe for one release cycle to see
> if anyone cribs.
>
> So I'm standing with the guy w/ asbestos short on. I'd like to see that OE
> err on the side of "do no harm" to existing users. Its hard enough to rally
> the troops to move to updated packages much less updated meta without you
> leaving perfectly reasonable versions of software out of oe-core.
>
> mike
>
>


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 22:01                               ` C Michael Sundius
@ 2011-01-19 22:22                                 ` Philip Balister
  0 siblings, 0 replies; 46+ messages in thread
From: Philip Balister @ 2011-01-19 22:22 UTC (permalink / raw)
  To: openembedded-devel

On 01/19/2011 02:01 PM, C Michael Sundius wrote:
> in rereading this I don't want to seem ungrateful, since we've certainly
> benefited from the great effort on everyones part (package and distro
> maintainers, yacto and OE... everyone).

Mike, well thought out replies from people using OE are always welcome. 
I think you made some really good points that I completely agree with.

Thanks,

Philip

>
> On Wed, Jan 19, 2011 at 1:55 PM, C Michael Sundius<msundius@sundius.com>wrote:
>
>>
>> It seems to me that this is a bit of a battle between the package
>> maintainers and the distro maintainers.. Looking at this from my managements
>> side of things, we use OE as a tool and its really just a means to the end.
>> our customers demand that we do not change things (versions of software),
>> they demand stability and they view a change in busybox or anything else a
>> threat to stability. our management has also made an edict that we can not
>> use gplv3. For completely non technical reasons we simply cannot move to new
>> package versions without a substantial business justification. I suspect
>> that that there are many (more than you realize) folk out there who are
>> using OE for their own distro. If you simply whack package versions because
>> something newer came out you will have these people maintaining separate
>> recipes and we'll be swamped with the load and this tool will loose one of
>> its best attributes.
>>
>> The comment that disturbed me was that distros should just move ahead
>> "because its making things hard for the package maintainer". That doesn't
>> wash with me because if people are using your package then you should
>> support it or let someone else be the maintainer. In essence the distro's
>> use of that package are your customers and the reason you have a job. OE
>> does not exist as a product, rather a tool that enables customers, you can't
>> create this in a vacuum without understanding who is using it.
>>
>> distro maintainers are not all dumb and if they are they'll be the last
>> single one using an outdated version of the software. When that happens a
>> smart package maintainer will call it out leave out the old package.
>> Further, it would be nice for a warning to take place so that it might have
>> a "depracated" tag associated with the recipe for one release cycle to see
>> if anyone cribs.
>>
>> So I'm standing with the guy w/ asbestos short on. I'd like to see that OE
>> err on the side of "do no harm" to existing users. Its hard enough to rally
>> the troops to move to updated packages much less updated meta without you
>> leaving perfectly reasonable versions of software out of oe-core.
>>
>> mike
>>
>>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 21:55                             ` C Michael Sundius
  2011-01-19 22:01                               ` C Michael Sundius
@ 2011-01-19 22:23                               ` Khem Raj
  2011-01-19 22:46                                 ` C Michael Sundius
  2011-01-20 10:20                                 ` Frans Meulenbroeks
  1 sibling, 2 replies; 46+ messages in thread
From: Khem Raj @ 2011-01-19 22:23 UTC (permalink / raw)
  To: openembedded-devel

On Wed, Jan 19, 2011 at 1:55 PM, C Michael Sundius <msundius@sundius.com> wrote:
> It seems to me that this is a bit of a battle between the package
> maintainers and the distro maintainers.. Looking at this from my managements
> side of things, we use OE as a tool and its really just a means to the end.
> our customers demand that we do not change things (versions of software),
> they demand stability and they view a change in busybox or anything else a
> threat to stability. our management has also made an edict that we can not
> use gplv3. For completely non technical reasons we simply cannot move to new
> package versions without a substantial business justification. I suspect
> that that there are many (more than you realize) folk out there who are
> using OE for their own distro. If you simply whack package versions because
> something newer came out you will have these people maintaining separate
> recipes and we'll be swamped with the load and this tool will loose one of
> its best attributes.
>
> The comment that disturbed me was that distros should just move ahead
> "because its making things hard for the package maintainer". That doesn't
> wash with me because if people are using your package then you should
> support it or let someone else be the maintainer.

I think keeping packages forever is not going to work either. If your
customer is someone with EOL of 20 years
OE should not keep versions you need just for you unless you invest
efforts in keeping that packages uptodate
thats why releases are for.

In essence the distro's
> use of that package are your customers and the reason you have a job. OE
> does not exist as a product, rather a tool that enables customers, you can't
> create this in a vacuum without understanding who is using it.
>

yes and assuming a package is being used by someone is also not the
right thing. IMO for products you should
base off on a release and maintain that for however long you wish and
give some space to OE to develop. Extreme on
both ends are unwanted thats why we are discussing what versions to
keep. Keeping every possible version around has
a cost and I believe if the recipe is not actively being tested its a
bit-rot. So we are trying to reduce that burden on OE devs
at the same time increase the quality of recipes.

> distro maintainers are not all dumb and if they are they'll be the last
> single one using an outdated version of the software. When that happens a
> smart package maintainer will call it out leave out the old package.
> Further, it would be nice for a warning to take place so that it might have
> a "depracated" tag associated with the recipe for one release cycle to see
> if anyone cribs.


recipes move into obsolete/ or nonworking/ dirs thats enough of a warning imo

>
> So I'm standing with the guy w/ asbestos short on. I'd like to see that OE
> err on the side of "do no harm" to existing users. Its hard enough to rally
> the troops to move to updated packages much less updated meta without you
> leaving perfectly reasonable versions of software out of oe-core.
>
> mike
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 22:23                               ` Khem Raj
@ 2011-01-19 22:46                                 ` C Michael Sundius
  2011-01-20 10:20                                 ` Frans Meulenbroeks
  1 sibling, 0 replies; 46+ messages in thread
From: C Michael Sundius @ 2011-01-19 22:46 UTC (permalink / raw)
  To: openembedded-devel

As with anything Balance and compromise is what make the best projects, I
don't think I'd want ever recipe to stay forever be not useful to anyone,
but I've appreciated the efforts that OE has made (since I've been using it)
to generally not break things for people who have invested time in using it.

On Wed, Jan 19, 2011 at 2:23 PM, Khem Raj <raj.khem@gmail.com> wrote:

> On Wed, Jan 19, 2011 at 1:55 PM, C Michael Sundius <msundius@sundius.com>
> wrote:
> > It seems to me that this is a bit of a battle between the package
> > maintainers and the distro maintainers.. Looking at this from my
> managements
> > side of things, we use OE as a tool and its really just a means to the
> end.
> > our customers demand that we do not change things (versions of software),
> > they demand stability and they view a change in busybox or anything else
> a
> > threat to stability. our management has also made an edict that we can
> not
> > use gplv3. For completely non technical reasons we simply cannot move to
> new
> > package versions without a substantial business justification. I suspect
> > that that there are many (more than you realize) folk out there who are
> > using OE for their own distro. If you simply whack package versions
> because
> > something newer came out you will have these people maintaining separate
> > recipes and we'll be swamped with the load and this tool will loose one
> of
> > its best attributes.
> >
> > The comment that disturbed me was that distros should just move ahead
> > "because its making things hard for the package maintainer". That doesn't
> > wash with me because if people are using your package then you should
> > support it or let someone else be the maintainer.
>
> I think keeping packages forever is not going to work either. If your
> customer is someone with EOL of 20 years
> OE should not keep versions you need just for you unless you invest
> efforts in keeping that packages uptodate
> thats why releases are for.
>
> In essence the distro's
> > use of that package are your customers and the reason you have a job. OE
> > does not exist as a product, rather a tool that enables customers, you
> can't
> > create this in a vacuum without understanding who is using it.
> >
>
> yes and assuming a package is being used by someone is also not the
> right thing. IMO for products you should
> base off on a release and maintain that for however long you wish and
> give some space to OE to develop. Extreme on
> both ends are unwanted thats why we are discussing what versions to
> keep. Keeping every possible version around has
> a cost and I believe if the recipe is not actively being tested its a
> bit-rot. So we are trying to reduce that burden on OE devs
> at the same time increase the quality of recipes.
>
> > distro maintainers are not all dumb and if they are they'll be the last
> > single one using an outdated version of the software. When that happens a
> > smart package maintainer will call it out leave out the old package.
> > Further, it would be nice for a warning to take place so that it might
> have
> > a "depracated" tag associated with the recipe for one release cycle to
> see
> > if anyone cribs.
>
>
> recipes move into obsolete/ or nonworking/ dirs thats enough of a warning
> imo
>
> >
> > So I'm standing with the guy w/ asbestos short on. I'd like to see that
> OE
> > err on the side of "do no harm" to existing users. Its hard enough to
> rally
> > the troops to move to updated packages much less updated meta without you
> > leaving perfectly reasonable versions of software out of oe-core.
> >
> > mike
> > _______________________________________________
> > Openembedded-devel mailing list
> > Openembedded-devel@lists.openembedded.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
> >
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19 22:23                               ` Khem Raj
  2011-01-19 22:46                                 ` C Michael Sundius
@ 2011-01-20 10:20                                 ` Frans Meulenbroeks
  2011-01-20 11:04                                   ` Graeme Gregory
  1 sibling, 1 reply; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-20 10:20 UTC (permalink / raw)
  To: openembedded-devel

Responding to both Khem's and Mike's replies.

2011/1/19 Khem Raj <raj.khem@gmail.com>:
> On Wed, Jan 19, 2011 at 1:55 PM, C Michael Sundius <msundius@sundius.com> wrote:
>> It seems to me that this is a bit of a battle between the package
>> maintainers and the distro maintainers.. Looking at this from my managements
>> side of things, we use OE as a tool and its really just a means to the end.
>> our customers demand that we do not change things (versions of software),
>> they demand stability and they view a change in busybox or anything else a
>> threat to stability. our management has also made an edict that we can not
>> use gplv3. For completely non technical reasons we simply cannot move to new
>> package versions without a substantial business justification. I suspect
>> that that there are many (more than you realize) folk out there who are
>> using OE for their own distro. If you simply whack package versions because
>> something newer came out you will have these people maintaining separate
>> recipes and we'll be swamped with the load and this tool will loose one of
>> its best attributes.

I understand your problem. Actually our situation is somewhat similar to yours.
However we've solved this in a much more rigid way.
Basically we pin a version of OE for a (range of) products.
We have an overlay that contains backports of newer recipes and
bugfixes that we need, but we will not upgrade automatically to a new
busybox release and not even take a new patch if it is not needed.
And if it is needed it will only be done after reviewing and testing
the changes.
The philosophy is "if it ain't broken, don't fix it", so if a patch
for busybox comes along that solves a problem we do not have (e.g.
because we do not use the applet) it will not get in.

As such some of our products still use busybox 1.13.2, and actually I
had to resolve a small isssue with it the other day.
upstream will probably laugh their pants off when I submit a bug
report for such an old version, and I do not expect OE to support this
version.
If we have a problem using this version we either find a patch that we
can backport, move to a new version or patch locally. What happens
depends on the issue at hand.
(and if we solve ourself and the problem is still in the latest
version we submit the patch upstream).

>>
>> The comment that disturbed me was that distros should just move ahead
>> "because its making things hard for the package maintainer". That doesn't
>> wash with me because if people are using your package then you should
>> support it or let someone else be the maintainer.

I only partially agree with that.
You cannot expect from a package maintainer to maintain a large number
of versions. There are definitely reasons to support more than one
version (especially for a lib)., but for me as a package maintainer it
more or less stops where upstream stops. E.g. if a package is upstream
at version X.5 and does not support X.3 any more and OE has both X.3
and X.5, then as a package maintainer I'm more than happy to peek into
issues with X.5 but I am not too interested to backport patches to
X.0, X.1, X.2, X.3 and X.4 or resolve problems in these releases.
Sorry to sound so rigid, but (generally speaking) if you have a
problem in a version that upstream does not support any more, and the
problem is solved in a version supported by upstream and this upstream
version is in OE, I am not too sure if you can expect from a package
maintainer to resolve it for you.
After all we're all just volunteers and for most of the people time is limited.

And if you feel you can do better wrt maintainership, feel free to let
me know which of the recipes I maintain, you want to take over.
Or alternately, pay me to support the version that you want to keep around.
>
> I think keeping packages forever is not going to work either. If your
> customer is someone with EOL of 20 years
> OE should not keep versions you need just for you unless you invest
> efforts in keeping that packages uptodate
> thats why releases are for.

releases and SCM.
fully agree.
>
> In essence the distro's
>> use of that package are your customers and the reason you have a job. OE

ROTFLOL. job. I think for most of the contributors (including me) no
one pays for the effort spent
Most of the work in OE is done by volunteers!
Basically most of the things I maintain is because I needed them for
some reason, wrote the recipe and submitted them (or updated and
submitted an orphaned recipe that I needed).
This is done as a service to the public and to give back to the
community, but I do not see it as a job, but more as a gift.

>> does not exist as a product, rather a tool that enables customers, you can't
>> create this in a vacuum without understanding who is using it.
>>
>
> yes and assuming a package is being used by someone is also not the
> right thing. IMO for products you should
> base off on a release and maintain that for however long you wish and
> give some space to OE to develop. Extreme on
> both ends are unwanted thats why we are discussing what versions to
> keep. Keeping every possible version around has
> a cost and I believe if the recipe is not actively being tested its a
> bit-rot. So we are trying to reduce that burden on OE devs
> at the same time increase the quality of recipes.

Agree.

And as said before, as a package maintainer I try to follow upstream
and support what upstream supports.
I do not want to take over the role of upstream wrt the package maintenance.
As a package maintainer I feel it as my task to make sure the version
from upstream builds in the OE environment, and ideally patches are
only there to deal with cross build issues, although every once in a
while a local bugfix is added.
That is how I see the role of a package maintainer, but others might
see things differently.
(btw for toolchain related things I can imagine we want to retain
older versions for a longer time, especially when there are functional
changes (I'm especially thinking about autohell))
And apologies if I sketch the things black and white, but at least
that makes things clear.
But also when it becomes to package maintainers tasks there is a gray area.
>
>> distro maintainers are not all dumb and if they are they'll be the last

This is not always in line with my experience. E.g. for mythtv one
distro maintainer for a while refused to go to a newer version because
the communication between client and backend changed and his mythbuntu
box or debian system or so was not yet using that new version.
Mythtv is not our most complicated recipe, but it is also not a
trivial one. The burden of maintaining one version takes more than
enough time, and I really didn't have the time, resources and
motivation to backport all metadata changes to the previous version.

>> single one using an outdated version of the software. When that happens a
>> smart package maintainer will call it out leave out the old package.
>> Further, it would be nice for a warning to take place so that it might have
>> a "depracated" tag associated with the recipe for one release cycle to see
>> if anyone cribs.
>
>
> recipes move into obsolete/ or nonworking/ dirs thats enough of a warning imo

If we want to fully support that then we should never use git mv to
create a new version, but (assuming the old version will be gone) move
the old version to obsolete or so and create a new recipe (of course
probably by starting with a copy of the old one). This is not the
current practice.
>
>>
>> So I'm standing with the guy w/ asbestos short on. I'd like to see that OE
>> err on the side of "do no harm" to existing users. Its hard enough to rally

I understand your problem, but that will eventually halt the system.
Sometimes disruptive changes are unavoidable (unless we want to keep
10 years of history or so).
The solution here is pinning on a release or a git rev. Then no one
will harm you.
You can't keep the cake and eat it too!

>> the troops to move to updated packages much less updated meta without you
>> leaving perfectly reasonable versions of software out of oe-core.

Best regards, Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-20 10:20                                 ` Frans Meulenbroeks
@ 2011-01-20 11:04                                   ` Graeme Gregory
  2011-01-20 13:16                                     ` Frans Meulenbroeks
  0 siblings, 1 reply; 46+ messages in thread
From: Graeme Gregory @ 2011-01-20 11:04 UTC (permalink / raw)
  To: openembedded-devel

On 20/01/2011 10:20, Frans Meulenbroeks wrote:
>
> And as said before, as a package maintainer I try to follow upstream
> and support what upstream supports.
> I do not want to take over the role of upstream wrt the package maintenance.
> As a package maintainer I feel it as my task to make sure the version
> from upstream builds in the OE environment, and ideally patches are
> only there to deal with cross build issues, although every once in a
> while a local bugfix is added.
> That is how I see the role of a package maintainer, but others might
> see things differently.
> (btw for toolchain related things I can imagine we want to retain
> older versions for a longer time, especially when there are functional
> changes (I'm especially thinking about autohell))
> And apologies if I sketch the things black and white, but at least
> that makes things clear.
> But also when it becomes to package maintainers tasks there is a gray area.
>
But this is part of the issue, this sort of working is fine for packages
you maintain, but you try and force your opinions on which packages are
relevant to other package maintainers. Go forth and do what you want
with your own little area but at least give some consideration that I
might actually be maintaining the recipes you class as junk.

Graeme




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-20 11:04                                   ` Graeme Gregory
@ 2011-01-20 13:16                                     ` Frans Meulenbroeks
  0 siblings, 0 replies; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-20 13:16 UTC (permalink / raw)
  To: openembedded-devel

2011/1/20 Graeme Gregory <dp@xora.org.uk>:
> On 20/01/2011 10:20, Frans Meulenbroeks wrote:
>>
>> And as said before, as a package maintainer I try to follow upstream
>> and support what upstream supports.
>> I do not want to take over the role of upstream wrt the package maintenance.
>> As a package maintainer I feel it as my task to make sure the version
>> from upstream builds in the OE environment, and ideally patches are
>> only there to deal with cross build issues, although every once in a
>> while a local bugfix is added.
>> That is how I see the role of a package maintainer, but others might
>> see things differently.
>> (btw for toolchain related things I can imagine we want to retain
>> older versions for a longer time, especially when there are functional
>> changes (I'm especially thinking about autohell))
>> And apologies if I sketch the things black and white, but at least
>> that makes things clear.
>> But also when it becomes to package maintainers tasks there is a gray area.
>>
> But this is part of the issue, this sort of working is fine for packages
> you maintain, but you try and force your opinions on which packages are
> relevant to other package maintainers. Go forth and do what you want
> with your own little area but at least give some consideration that I
> might actually be maintaining the recipes you class as junk.

Ehm, no. I don't want to enforce things onto others, and if you feel
you want to maintain 5 versions be my guest (assuming you maintain
them well)
And it is not up to me to give a classification of the work of others.
Then again I do not like it if people force maintenance work onto me
by pinning a version that I as a package maintainer feel should be
expired (for good reasons).

The real issues at hand are:
- we have quite a lot of recipes that were added at some point in time
but that are not maintained, apart from maybe sometimes getting a
small bug fix from someone actually passing by or caused by a global
change (e.g. adding md5sum's)
- maintainership of recipes is often unclear, so if there is an issue
it is unclear who to contact. To give an example: openssl: the last 15
commits in recipes/openssl are done by 10 different people.
MAINTAINERS does not list a maintainer for openssl. (and yeah, I know
I can send to the ML or contact the last committer)
- some maintainers don't do a very good job in maintaining, especially
when it comes to expiring old stuff, leaving a recipe that is not
fetchable, does not patch or does not build (sometimes because an .inc
file was modified and the patch works on the latest version, but older
versions are build let alone tested). I seem to recall a list from
Martin with some 1000 recipes on it that did not fetch or patch.
(actually not sure about the number). Updating also seems to be
lacking. Some action was taken then, but if I recall correctly this
was never completed.I

To get back at how well things are maintained, let us as an example
look at openssl again.

I've expired a number of versions that had known security
vulnerabilities last august. This gave a lot of flak as I accidently
deleted a version that was pinned (openssl_0.9.8m.bb). One of the
things observed at that time was that this version had CVE's and that
newer versions were available (9n and 9o). Currently (almost half a
year later) openssl is at 0.9.8q but OE is still at 0.9.8m.
I would have expected a maintainer would have taken action (and yes in
case of CVE's, I would suggest that the old versions with the security
issues are retired).
And I know way too well that we all are busy and have limited time,
but I would have hoped that someone would have taken action. And no,
it is not going to be me, I took enough heat on openssl last summer
and decided not to touch the recipe with a pole, but I would have
appreciated it that the people who gave me a hard time then at least
would have brought the recipe to 9o.
I explicitly mentioned .9o on aug 16.

Quote:
seems a good plan to upgrade to 0.9.8o though as it fixes some
security vulnerabilities

But that did not lead to action and at that point it seemed better for
me not to touch openssl at all (although I feel a security
vulnerability in something like openssl is quite serious).

And this is something that starts to irritate me in OE. apparently
there is always time to shoot the messenger who comes with suggestions
on how to improve, and to bash at people that at least try to resolve
some issues but the silence is deafening when it comes to actually
doing things.

And apologies if I sound a little bit sour.

Frans.



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  9:25                 ` Frans Meulenbroeks
@ 2011-01-22 18:16                   ` Tom Rini
  2011-01-23 10:11                     ` Frans Meulenbroeks
  0 siblings, 1 reply; 46+ messages in thread
From: Tom Rini @ 2011-01-22 18:16 UTC (permalink / raw)
  To: openembedded-devel

On 01/19/2011 02:25 AM, Frans Meulenbroeks wrote:
> 2011/1/19 Frans Meulenbroeks<fransmeulenbroeks@gmail.com>:
>> 2011/1/18 Koen Kooi<k.kooi@student.utwente.nl>:
>>   if we make the delta between the 3 be just
>>>> the SRC_URI + checksums?
>>>
>>> Something like:
>>>
>>> * keep gplv2 around if upstream switched to gplv3 at some point
>>
>> agree; as said before suggest to reflect the switch in the name of the recipe.
>>
>>> * allow old stable, stable and dev (e.g. glib 2.24, 2.26 and 2.27+git)
>>
>> I'd say retire old stable after a while (e.g. after stable is there
>> for half a year, unless of course there are compelling reasons to keep
>> it).
>> Wrt dev: if we want to go to a maintainers model similar to u-boot and
>> linux, I would expect this to live in a separate branch or layer.
>> Once proven stable the maintainers pulls it into the core archive.
>>
>>> * allow active contributers some leeway to test multiple subreleases
>>> (e.g. busybox 1.18.[0-99]
>>
>> Again here, I feel it is much better to put it into a different branch
>> or layer, and once stable it is pulled into the mainline.
>> Again similar to what Linus and Wolfgang do.
>>
>>> * The maintainer can have as many versions as he wants to.
>>
>> my preference: not in the main layer.
>>
>>> * toolchain is exempt from all that since having 20 binutils versions is
>>> sometimes needed when building for 20 different archs[1].
>>
>> Fully agree.
>>>
>>> But I seriously think we are overengineering this. We should have people
>>> actually working on oe-core and meta-oe reach a consensus when
>>> encountering problems.
>>
>> I beg to disagree on this.
>> We have a unique opportunity at hand to improve, so it seems wise to
>> raise the bar quality wise.
>>
>> E.g. wrt what Richard wrote in the start post
>>
>> [quote]
>> * Creation of a subset of metadata that has a consistent high quality
>>   standard:
>>    - removal of legacy components (pkgconfig hacks, libtool hacks,
>>      legacy staging, unneeded compiler arguments)
>>    - consistent metadata layout, spacing, use of variables
>>    - migration away from outdated practises (e.g. use BBCLASSEXTEND)
>> [/quote]
>>
>> I feel when migrating recipes we should also assure that they adhere
>> to a certain quality standard. That would need us to define a minimal
>> standard though.
>> Currently I feel we are just moving recipes, only fixing what is
>> really needed to get things running (like removing legacy staging)
>> whereas we could do much better.
>> There are currently some things in OE that could use some improvement
>> and we should take the opportunity to improve.l

Here's what I worry about.  It shouldn't take an overly active 
maintainer for most recipes.  Sure, some programs (and of course the 
toolchain) require some care and knowledge.  But lots of things don't. 
I fear we're trying to get to the point where if someone is going to 
submit a recipe for some program they need to sign up to make sure it's 
always pointing to the most up to date upstream and other mandates.  One 
of the strengths of OE today is that it's not overly burdensome to 
submit changes back and have them accepted.

>>
>> And yes, I have no problem in spending time to improve things, but if
>> there is no consensus on where we should go this is not going to be
>> effective (and actually makes it harder to improve quality wise).
>> To take the bikeshed analogy up again. It does not really help if
>> everyone start to paint in his own favourite color (unless maybe if
>> you are still stuck in the flower power era).
>>
>> Frans
>>
> I know it is fairly schizofrenic to reply on ones own post, but I just
> peeked into the meta-oe layer and found an excellent case on what we
> should try to avoid when bringing over recipes from oe to meta-oe.
> path: root/recipes-graphics/directfb
> Mode	Name	Size	
> d---------	++dfb	48	logplain
> -rw-r--r--	++dfb_1.0.0.bb	588	logplain
> d---------	directfb-1.2.8	50	logplain
> d---------	directfb-1.4.6	41	logplain
> -rw-r--r--	directfb-examples_1.0.0.bb	406	logplain
> -rw-r--r--	directfb-examples_1.2.0.bb	409	logplain
> -rw-r--r--	directfb.inc	2038	logplain
> -rw-r--r--	directfb_1.4.6.bb	615	logplain
> d---------	files	415	logplain
> d---------	fusionsound-1.1.0+git20070709	54	logplain
> -rw-r--r--	fusionsound_1.1.0+git20070709.bb	1479	logplain
> -rw-r--r--	lite_0.9.26+cvs20070207.bb	1140	logplain
>
> Why is there a directfb-examples_1.0.0.bb? In OE no-one is pinning
> this version, and given the fact that these are examples there is
> probably little reason to keep it.

They are example programs for directfb which also means they are handy 
demo programs for "hey, is directfb working on my platform and looking 
right".

> Something similar for directfb-1.2.8. Maybe I overlooked things but I
> could not find a the user of this dir. I guess it could be gone.

I suppose it could be updated to 1.2.10, yes.  And directfb is enough by 
itself, although it would be nice if we went and took a stab at making 
it an option to use rather than X, when possible again.

> If we want to improve on quality it is probably not a bad idea to at
> least have a checklist with improvement actions that should be done
> when migrating recipes (e.g. remove stale patches, unused&  unpinned
> older versions etc etc). I know it takes time, but it can also
> increase the overall quality. Let's take that opportunity.

OK, but what's wrong in this directory?  It's on known to work for 
someone versions, I don't see legacy staging and iirc the pkg-config 
changes aren't hacks but fixing upstream.

-- 
Tom Rini
Mentor Graphics Corporation



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-19  8:46               ` Frans Meulenbroeks
  2011-01-19  8:52                 ` Graeme Gregory
  2011-01-19 16:56                 ` Khem Raj
@ 2011-01-22 18:20                 ` Tom Rini
  2011-01-23  2:05                   ` Otavio Salvador
  2 siblings, 1 reply; 46+ messages in thread
From: Tom Rini @ 2011-01-22 18:20 UTC (permalink / raw)
  To: openembedded-devel

On 01/19/2011 01:46 AM, Frans Meulenbroeks wrote:
> 2011/1/18 Khem Raj<raj.khem@gmail.com>:
>> On Tue, Jan 18, 2011 at 12:48 PM, Tom Rini<tom_rini@mentor.com>  wrote:
>>> I think we also agree here.  But what's the rule of thumb(s) we want to
>>> have, to provide enough choice without too much headache?  As I said
>>> elsewhere, .inc files should probably be used a whole lot more, to help with
>>> the problem of recipe bugfixing and N recipes for an app with the problem.
>>>   We should probably also say that in addition to the "keep the last GPLv2+
>>> version around" rule of thumb we should also have a "keep the latest stable
>>> release" around too.  But what else?  To use busybox as an example, do we
>>> really need to keep 1.18.0 and 1.18.1 around when we have 1.18.2?  How about
>>> if we make the delta between the 3 be just the SRC_URI + checksums?
>>
>>
>> Well what to keep around can not be generalized so much. It also
>> depends upon the nature of releases for various packages
>> some packages have a major release and then push out bug fix releases
>> like busy box case you sited but there are packages
>> which only do major releases which can cause compatibility hassles as
>> new interfaces come and old ones are removed etc.
>> so I think it has to be flexible and mostly left to package
>> maintainers discretion as they know the nature of releases most but
>> I like the idea of keeping one GPLv2 release around and 1 latest
>> release around. If a distro pinned a version then that should
>> be considered as well. It is bad to sweep underneath a distros
>> pinnings. Then they have to rebuild and they get problems
>> as they have to make sure that if a package bump happens then they
>> should be able to define an upgrade path
>>
>> Sometimes newer is not always better in case of udev the size has
>> increased over the time and some distro's may not like
>> it and there can be many such scenarios.
>
> I wholehearthy agree with the proposal that it is left to the package
> maintainers discretion.

And what about packages that have been, well, gently orphaned?  I think 
we need to accept and understand that there will be a class of recipes 
that are simple enough that they don't need nor have spelled out 
maintainers.

> Wrt the GPLv2+ version. I suggest to reflect this in the name.
> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
> samba-gplv3). Then it immediately becomes obvious why there are two
> versions.

Put me in the "no" camp here please, we have a LICENSE field for a reason.

-- 
Tom Rini
Mentor Graphics Corporation



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-22 18:20                 ` Tom Rini
@ 2011-01-23  2:05                   ` Otavio Salvador
  0 siblings, 0 replies; 46+ messages in thread
From: Otavio Salvador @ 2011-01-23  2:05 UTC (permalink / raw)
  To: openembedded-devel

On Sat, Jan 22, 2011 at 16:20, Tom Rini <tom_rini@mentor.com> wrote:
> On 01/19/2011 01:46 AM, Frans Meulenbroeks wrote:
>> Wrt the GPLv2+ version. I suggest to reflect this in the name.
>> E.g. you could have samba and samba-gplv2 (or perhaps samba-gplv2 and
>> samba-gplv3). Then it immediately becomes obvious why there are two
>> versions.
>
> Put me in the "no" camp here please, we have a LICENSE field for a reason.

+1

What we'd do for multi-license projects? foo-lgplv2-gplv2-mit-bsd.bb? grrr

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-22 18:16                   ` Tom Rini
@ 2011-01-23 10:11                     ` Frans Meulenbroeks
  2011-01-24 17:45                       ` Tom Rini
  0 siblings, 1 reply; 46+ messages in thread
From: Frans Meulenbroeks @ 2011-01-23 10:11 UTC (permalink / raw)
  To: openembedded-devel

2011/1/22 Tom Rini <tom_rini@mentor.com>:
> On 01/19/2011 02:25 AM, Frans Meulenbroeks wrote:
>>
>> 2011/1/19 Frans Meulenbroeks<fransmeulenbroeks@gmail.com>:
>>>
>>> 2011/1/18 Koen Kooi<k.kooi@student.utwente.nl>:
>>>  if we make the delta between the 3 be just
>>>>>
>>>>> the SRC_URI + checksums?
>>>>
>>>> Something like:
>>>>
>>>> * keep gplv2 around if upstream switched to gplv3 at some point
>>>
>>> agree; as said before suggest to reflect the switch in the name of the
>>> recipe.
>>>
>>>> * allow old stable, stable and dev (e.g. glib 2.24, 2.26 and 2.27+git)
>>>
>>> I'd say retire old stable after a while (e.g. after stable is there
>>> for half a year, unless of course there are compelling reasons to keep
>>> it).
>>> Wrt dev: if we want to go to a maintainers model similar to u-boot and
>>> linux, I would expect this to live in a separate branch or layer.
>>> Once proven stable the maintainers pulls it into the core archive.
>>>
>>>> * allow active contributers some leeway to test multiple subreleases
>>>> (e.g. busybox 1.18.[0-99]
>>>
>>> Again here, I feel it is much better to put it into a different branch
>>> or layer, and once stable it is pulled into the mainline.
>>> Again similar to what Linus and Wolfgang do.
>>>
>>>> * The maintainer can have as many versions as he wants to.
>>>
>>> my preference: not in the main layer.
>>>
>>>> * toolchain is exempt from all that since having 20 binutils versions is
>>>> sometimes needed when building for 20 different archs[1].
>>>
>>> Fully agree.
>>>>
>>>> But I seriously think we are overengineering this. We should have people
>>>> actually working on oe-core and meta-oe reach a consensus when
>>>> encountering problems.
>>>
>>> I beg to disagree on this.
>>> We have a unique opportunity at hand to improve, so it seems wise to
>>> raise the bar quality wise.
>>>
>>> E.g. wrt what Richard wrote in the start post
>>>
>>> [quote]
>>> * Creation of a subset of metadata that has a consistent high quality
>>>  standard:
>>>   - removal of legacy components (pkgconfig hacks, libtool hacks,
>>>     legacy staging, unneeded compiler arguments)
>>>   - consistent metadata layout, spacing, use of variables
>>>   - migration away from outdated practises (e.g. use BBCLASSEXTEND)
>>> [/quote]
>>>
>>> I feel when migrating recipes we should also assure that they adhere
>>> to a certain quality standard. That would need us to define a minimal
>>> standard though.
>>> Currently I feel we are just moving recipes, only fixing what is
>>> really needed to get things running (like removing legacy staging)
>>> whereas we could do much better.
>>> There are currently some things in OE that could use some improvement
>>> and we should take the opportunity to improve.l
>
> Here's what I worry about.  It shouldn't take an overly active maintainer
> for most recipes.  Sure, some programs (and of course the toolchain) require
> some care and knowledge.  But lots of things don't. I fear we're trying to
> get to the point where if someone is going to submit a recipe for some
> program they need to sign up to make sure it's always pointing to the most
> up to date upstream and other mandates.  One of the strengths of OE today is
> that it's not overly burdensome to submit changes back and have them
> accepted.

Tom, good points.

We should not make it too difficult to submit recipes (but I would
appreciate if submitted recipes adhere to some to-be-defined minimal
set of QA rules).
But it would help to know if a recipe has an active maintainer or not.
I think I would be more inclined to fix issues or update the recipe in
case it is unmaintained.
If a recipe is maintained, I would probably first see if the
maintainer takes action (and trigger him/her). It also makes clear who
to contact  if there is an issue. Now it is not always too clear who
is working on what).

The issue is also that if you pick up a recipe that is (or seems)
unmaintained and update it, and make a mistake there is always someone
around the corner with some nasty remarks, making it not too
interesting to work on these recipes.

Wrt your coment on "submit changes back and have them accepted". I
fear I see this not as a strength. The process is ok, but it does not
work too well for those unmaintained recipes.
Typically the unmaintained recipes are in some corner of OE and not
many people care about those. So if you submit a patch to these, it is
quite common to get no feedback at all.
A policy saying: submit patch, wait two weeks for feedback, if none
given, send a ping, wait another two weeks still no feedback, does not
work too well (but the scenario I sketchted happens too often.
Now after this process I feel I can apply the patch (with 4 weeks
delay, which seems quite a lot if the patch is small), but if the
submitter has no commit rights it'll just land in patchwork waiting
for someone to pick things up.
Especially the latter scenario will cause that that person submitting
the patch will give up after submitting one or two patches.
Maybe I'm drawing a black scenario, but this is what I see happen.
>
>>>
>>> And yes, I have no problem in spending time to improve things, but if
>>> there is no consensus on where we should go this is not going to be
>>> effective (and actually makes it harder to improve quality wise).
>>> To take the bikeshed analogy up again. It does not really help if
>>> everyone start to paint in his own favourite color (unless maybe if
>>> you are still stuck in the flower power era).
>>>
>>> Frans
>>>
>> I know it is fairly schizofrenic to reply on ones own post, but I just
>> peeked into the meta-oe layer and found an excellent case on what we
>> should try to avoid when bringing over recipes from oe to meta-oe.
>> path: root/recipes-graphics/directfb
>> Mode    Name    Size
>> d---------      ++dfb   48      logplain
>> -rw-r--r--      ++dfb_1.0.0.bb  588     logplain
>> d---------      directfb-1.2.8  50      logplain
>> d---------      directfb-1.4.6  41      logplain
>> -rw-r--r--      directfb-examples_1.0.0.bb      406     logplain
>> -rw-r--r--      directfb-examples_1.2.0.bb      409     logplain
>> -rw-r--r--      directfb.inc    2038    logplain
>> -rw-r--r--      directfb_1.4.6.bb       615     logplain
>> d---------      files   415     logplain
>> d---------      fusionsound-1.1.0+git20070709   54      logplain
>> -rw-r--r--      fusionsound_1.1.0+git20070709.bb        1479    logplain
>> -rw-r--r--      lite_0.9.26+cvs20070207.bb      1140    logplain
>>
>> Why is there a directfb-examples_1.0.0.bb? In OE no-one is pinning
>> this version, and given the fact that these are examples there is
>> probably little reason to keep it.
>
> They are example programs for directfb which also means they are handy demo
> programs for "hey, is directfb working on my platform and looking right".

I understand that. and I am not saying that the recipe should go. What
I wanted to say is that if we have a 1.2.0 version (of a recipe for
example progs) what is the point to keep also version1.0.0 around.
It is unpinned so I see little reason to keep 1.0.0. Of course 1.2.0
should be kept.
>
>> Something similar for directfb-1.2.8. Maybe I overlooked things but I
>> could not find a the user of this dir. I guess it could be gone.
>
> I suppose it could be updated to 1.2.10, yes.  And directfb is enough by
> itself, although it would be nice if we went and took a stab at making it an
> option to use rather than X, when possible again.

The issue at hand is that there is no recipe directfb 1.2.8 any more.
We only have 1.4.6. However the patch files for 1.2.8 are still left,
but there seem no users left of this dir.
So basically unused files have been moved to meta-oe. I doubt that
that is desirable.
>
>> If we want to improve on quality it is probably not a bad idea to at
>> least have a checklist with improvement actions that should be done
>> when migrating recipes (e.g. remove stale patches, unused&  unpinned
>> older versions etc etc). I know it takes time, but it can also
>> increase the overall quality. Let's take that opportunity.
>
> OK, but what's wrong in this directory?  It's on known to work for someone
> versions, I don't see legacy staging and iirc the pkg-config changes aren't
> hacks but fixing upstream.

As mentioned above there are files in it that are not used by any
recipe, it contains a recipe that is probably better removed (the
1.0.0 version of the examples, keeping the 1.2.0 version as the sole
version).
I can also imagine the files dir could be renamed or split (I haven't
really checked if the patches are for one recipe or shared, but I
suspect the former).
I did not peek into the recipes itself but I can imagine they could
also be given a more standard layout (oe-stylize).

Nothing dramatically, just some small changes that gives things a more
constent look (both at the directory level and at the content level).
I fee the move to meta-oe could be used to raise the bar a little bit.

Wrt the LICENSE field and the recipe names (and your next post)..
I was just coining an idea. I feel it is useful to have an indication
why there are multiple versions of a recipe instead of one.
Especially in the case of recipes that have no maintainer this seems
useful information, as in that case someone who wants to change the
recipe gets an idea why it is useful to fix the old recipe too.
(the options for such a person are: fix old recipe too, if no one uses
it, it is a waste of time, do not fix the recipe and leave a possibly
broken recipe, or remove the old recipe and risk that there is a
whiner who apparently does not care to fix the recipe, but does
complain if someone does a best effort attempt to fix things.

Encoding this in the name makes it very explicit what the reason is to
have two versions and reduces the chance of errors.

For gplv2/v3 LICENSE can be used, but that does not help if e.g. we
want to keep an old version of foo because the footprint is much
smaller.
So either we need a different name, a comment in the recipe or a
README like file describing the differences).

With LICENSE the chance is a little bit higher that someone accidently
overlooks that that is the reason for having two versions. Name
encoding makes it more visible.
But if people feels relying only on LICENSE because otherwise the
namespace becomes too cluttered, that is also fine with me.

Best regards, Frans



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Yocto Project and OE - Where now?
  2011-01-23 10:11                     ` Frans Meulenbroeks
@ 2011-01-24 17:45                       ` Tom Rini
  0 siblings, 0 replies; 46+ messages in thread
From: Tom Rini @ 2011-01-24 17:45 UTC (permalink / raw)
  To: openembedded-devel

On 01/23/2011 03:11 AM, Frans Meulenbroeks wrote:
> 2011/1/22 Tom Rini<tom_rini@mentor.com>:
[snip]
>> Here's what I worry about.  It shouldn't take an overly active maintainer
>> for most recipes.  Sure, some programs (and of course the toolchain) require
>> some care and knowledge.  But lots of things don't. I fear we're trying to
>> get to the point where if someone is going to submit a recipe for some
>> program they need to sign up to make sure it's always pointing to the most
>> up to date upstream and other mandates.  One of the strengths of OE today is
>> that it's not overly burdensome to submit changes back and have them
>> accepted.
>
> Tom, good points.
>
> We should not make it too difficult to submit recipes (but I would
> appreciate if submitted recipes adhere to some to-be-defined minimal
> set of QA rules).

That's fine, but again I want to point out that 95% of the time that 
means doing almost nothing special.

> But it would help to know if a recipe has an active maintainer or not.
> I think I would be more inclined to fix issues or update the recipe in
> case it is unmaintained.
> If a recipe is maintained, I would probably first see if the
> maintainer takes action (and trigger him/her). It also makes clear who
> to contact  if there is an issue. Now it is not always too clear who
> is working on what).

I have to disagree at least a little bit here.  I guess, if we're going 
to move towards more of a pull model we should try and move towards 
something like the kernel where yes, people submit changes (and not just 
bug reports) up along the chain.

> The issue is also that if you pick up a recipe that is (or seems)
> unmaintained and update it, and make a mistake there is always someone
> around the corner with some nasty remarks, making it not too
> interesting to work on these recipes.

So, OK.  Here's a problem.  I believe we must not make policy choices 
based on personality conflicts we're having within the community.  No, I 
don't know what we can do about this as I certainly see value in what 
everyone with write access is doing.  And No, I don't see killfiling 
people as a valid solution either.

> Wrt your coment on "submit changes back and have them accepted". I
> fear I see this not as a strength. The process is ok, but it does not
> work too well for those unmaintained recipes.
> Typically the unmaintained recipes are in some corner of OE and not
> many people care about those. So if you submit a patch to these, it is
> quite common to get no feedback at all.

And this is where I see people with general overall interest being in 
charge and applying stuff.

> A policy saying: submit patch, wait two weeks for feedback, if none
> given, send a ping, wait another two weeks still no feedback, does not
> work too well (but the scenario I sketchted happens too often.

I agree.  In fact, I wish more people felt empowered (or had the time) 
to say "this looks sane, applied".  But yes, this is also where that 
conflict sometimes comes into play.

> Now after this process I feel I can apply the patch (with 4 weeks
> delay, which seems quite a lot if the patch is small), but if the
> submitter has no commit rights it'll just land in patchwork waiting
> for someone to pick things up.
> Especially the latter scenario will cause that that person submitting
> the patch will give up after submitting one or two patches.
> Maybe I'm drawing a black scenario, but this is what I see happen.

I do agree that stuff isn't being applied quickly in some cases.

>>
>>>>
>>>> And yes, I have no problem in spending time to improve things, but if
>>>> there is no consensus on where we should go this is not going to be
>>>> effective (and actually makes it harder to improve quality wise).
>>>> To take the bikeshed analogy up again. It does not really help if
>>>> everyone start to paint in his own favourite color (unless maybe if
>>>> you are still stuck in the flower power era).
>>>>
>>>> Frans
>>>>
>>> I know it is fairly schizofrenic to reply on ones own post, but I just
>>> peeked into the meta-oe layer and found an excellent case on what we
>>> should try to avoid when bringing over recipes from oe to meta-oe.
>>> path: root/recipes-graphics/directfb
>>> Mode    Name    Size
>>> d---------      ++dfb   48      logplain
>>> -rw-r--r--      ++dfb_1.0.0.bb  588     logplain
>>> d---------      directfb-1.2.8  50      logplain
>>> d---------      directfb-1.4.6  41      logplain
>>> -rw-r--r--      directfb-examples_1.0.0.bb      406     logplain
>>> -rw-r--r--      directfb-examples_1.2.0.bb      409     logplain
>>> -rw-r--r--      directfb.inc    2038    logplain
>>> -rw-r--r--      directfb_1.4.6.bb       615     logplain
>>> d---------      files   415     logplain
>>> d---------      fusionsound-1.1.0+git20070709   54      logplain
>>> -rw-r--r--      fusionsound_1.1.0+git20070709.bb        1479    logplain
>>> -rw-r--r--      lite_0.9.26+cvs20070207.bb      1140    logplain
>>>
>>> Why is there a directfb-examples_1.0.0.bb? In OE no-one is pinning
>>> this version, and given the fact that these are examples there is
>>> probably little reason to keep it.
>>
>> They are example programs for directfb which also means they are handy demo
>> programs for "hey, is directfb working on my platform and looking right".
>
> I understand that. and I am not saying that the recipe should go. What
> I wanted to say is that if we have a 1.2.0 version (of a recipe for
> example progs) what is the point to keep also version1.0.0 around.
> It is unpinned so I see little reason to keep 1.0.0. Of course 1.2.0
> should be kept.

How much of a directfb person are you?  Dusting off some mental cobwebs 
here, but iirc, not everything one might want to do to show off directfb 
is ported up to the latest version.   ie you might well need 1.0.x in 
order to get the gtk+ (and then mozilla) stuff to the point where it 
does something now.  Yes, this probably isn't what everyone wants but 
it's useful in other cases.

>>> Something similar for directfb-1.2.8. Maybe I overlooked things but I
>>> could not find a the user of this dir. I guess it could be gone.
>>
>> I suppose it could be updated to 1.2.10, yes.  And directfb is enough by
>> itself, although it would be nice if we went and took a stab at making it an
>> option to use rather than X, when possible again.
>
> The issue at hand is that there is no recipe directfb 1.2.8 any more.
> We only have 1.4.6. However the patch files for 1.2.8 are still left,
> but there seem no users left of this dir.
> So basically unused files have been moved to meta-oe. I doubt that
> that is desirable.

Yeah, ok, a porting issue since mainline OE has 1.2.8 still.

>>> If we want to improve on quality it is probably not a bad idea to at
>>> least have a checklist with improvement actions that should be done
>>> when migrating recipes (e.g. remove stale patches, unused&    unpinned
>>> older versions etc etc). I know it takes time, but it can also
>>> increase the overall quality. Let's take that opportunity.
>>
>> OK, but what's wrong in this directory?  It's on known to work for someone
>> versions, I don't see legacy staging and iirc the pkg-config changes aren't
>> hacks but fixing upstream.
>
> As mentioned above there are files in it that are not used by any
> recipe, it contains a recipe that is probably better removed (the
> 1.0.0 version of the examples, keeping the 1.2.0 version as the sole
> version).
> I can also imagine the files dir could be renamed or split (I haven't
> really checked if the patches are for one recipe or shared, but I
> suspect the former).
> I did not peek into the recipes itself but I can imagine they could
> also be given a more standard layout (oe-stylize).
>
> Nothing dramatically, just some small changes that gives things a more
> constent look (both at the directory level and at the content level).
> I fee the move to meta-oe could be used to raise the bar a little bit.

OK, so your valid point here is that there's some work to bring it up to 
speed a bit more.  That's good and fine and I'll agree.  But the reason 
it's like this I imagine is that it's also really hard and not fun to 
have little to nothing building and trying to get everything perfect out 
of the gate isn't possible either.

> Wrt the LICENSE field and the recipe names (and your next post)..
> I was just coining an idea. I feel it is useful to have an indication
> why there are multiple versions of a recipe instead of one.
> Especially in the case of recipes that have no maintainer this seems
> useful information, as in that case someone who wants to change the
> recipe gets an idea why it is useful to fix the old recipe too.
> (the options for such a person are: fix old recipe too, if no one uses
> it, it is a waste of time, do not fix the recipe and leave a possibly
> broken recipe, or remove the old recipe and risk that there is a
> whiner who apparently does not care to fix the recipe, but does
> complain if someone does a best effort attempt to fix things.

I think this stems in part from another difference of opinion.  I say 
it's better to make a set of changes that've been not tested in every 
possible combination but tested in some / most cases.  So yes, you 
should always change all of the versions.  And say based on my 
libc-uclibc change be right about 85% of the time (which isn't as high 
as I would like sadly).  It's the dev branch and there's SCM.

> Encoding this in the name makes it very explicit what the reason is to
> have two versions and reduces the chance of errors.
>
> For gplv2/v3 LICENSE can be used, but that does not help if e.g. we
> want to keep an old version of foo because the footprint is much
> smaller.
> So either we need a different name, a comment in the recipe or a
> README like file describing the differences).

Changing the name introduces other overhead (like having to set 
PREFERRED_PROVIDERS) so yes, a comment (either in the file or where it's 
pinned) would be best.

> With LICENSE the chance is a little bit higher that someone accidently
> overlooks that that is the reason for having two versions. Name
> encoding makes it more visible.
> But if people feels relying only on LICENSE because otherwise the
> namespace becomes too cluttered, that is also fine with me.

Thanks :)

-- 
Tom Rini
Mentor Graphics Corporation



^ permalink raw reply	[flat|nested] 46+ messages in thread

end of thread, other threads:[~2011-01-24 17:46 UTC | newest]

Thread overview: 46+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-14 17:49 Yocto Project and OE - Where now? Richard Purdie
2011-01-17 15:15 ` Koen Kooi
2011-01-17 19:01 ` Frans Meulenbroeks
2011-01-18  7:21   ` Graeme Gregory
2011-01-18  8:05     ` Otavio Salvador
2011-01-18  8:47       ` Koen Kooi
2011-01-18  9:17         ` Richard Purdie
2011-01-18  9:12       ` Graeme Gregory
2011-01-18 10:15         ` Richard Purdie
2011-01-18 11:05           ` Frans Meulenbroeks
2011-01-18 11:31       ` Mark Brown
2011-01-18 18:45         ` Frans Meulenbroeks
2011-01-18 16:54       ` Tom Rini
2011-01-18 20:12         ` Koen Kooi
2011-01-18 20:48           ` Tom Rini
2011-01-18 22:05             ` Koen Kooi
2011-01-19  8:45               ` Frans Meulenbroeks
2011-01-19  8:58                 ` Graeme Gregory
2011-01-19  9:16                   ` Frans Meulenbroeks
2011-01-19  9:25                 ` Frans Meulenbroeks
2011-01-22 18:16                   ` Tom Rini
2011-01-23 10:11                     ` Frans Meulenbroeks
2011-01-24 17:45                       ` Tom Rini
2011-01-18 22:48             ` Khem Raj
2011-01-19  8:46               ` Frans Meulenbroeks
2011-01-19  8:52                 ` Graeme Gregory
2011-01-19  9:12                   ` Frans Meulenbroeks
2011-01-19  9:19                     ` Graeme Gregory
2011-01-19 11:31                     ` Joshua Lock
2011-01-19 12:44                       ` Frans Meulenbroeks
2011-01-19 15:09                         ` Mike Westerhof
2011-01-19 15:44                           ` Frans Meulenbroeks
2011-01-19 18:03                           ` Khem Raj
2011-01-19 21:55                             ` C Michael Sundius
2011-01-19 22:01                               ` C Michael Sundius
2011-01-19 22:22                                 ` Philip Balister
2011-01-19 22:23                               ` Khem Raj
2011-01-19 22:46                                 ` C Michael Sundius
2011-01-20 10:20                                 ` Frans Meulenbroeks
2011-01-20 11:04                                   ` Graeme Gregory
2011-01-20 13:16                                     ` Frans Meulenbroeks
2011-01-19 16:56                 ` Khem Raj
2011-01-22 18:20                 ` Tom Rini
2011-01-23  2:05                   ` Otavio Salvador
2011-01-19  7:47           ` Frans Meulenbroeks
2011-01-19  8:16             ` Martin Jansa

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.