OpenEmbedded and the Yocto Project

OpenEmbedded and the Yocto Project

[Philip Balister]

The Linux Foundation has announced the Yocto Project (based partly on 
OpenEmbedded), http://yoctoproject.org/.

The OpenEmbedded eV board is pleased to see more people using 
OpenEmbedded and would like to develop a long term partnership with the 
Yocto Project.

The eV board would like the OpenEmbedded Community to start discussing 
the Yocto Project and help develop a long term plan to advance the goals 
of both projects. This process should result in a Voting Proposal that 
the OpenEmbedded eV can vote on.

The eV is the only formal decision making body for OpenEmbedded. If you 
are a community member that would like a voice in the voting proposal, 
respond to Stefan Schmidt's call for eV members.

Thanks for your help,

Philip

Re: OpenEmbedded and the Yocto Project

[Koen Kooi]

Op 2 nov 2010, om 15:16 heeft Philip Balister het volgende geschreven:

> The Linux Foundation has announced the Yocto Project (based partly on OpenEmbedded), http://yoctoproject.org/.
> 
> The OpenEmbedded eV board is pleased to see more people using OpenEmbedded and would like to develop a long term partnership with the Yocto Project.
> 
> The eV board would like the OpenEmbedded Community to start discussing the Yocto Project and help develop a long term plan to advance the goals of both projects. This process should result in a Voting Proposal that the OpenEmbedded eV can vote on.

Before we go vote on stuff, I propose to try to work on relatively small focussed projects together to see how our goal and way of working match the yoctoproject ones. Things like tinderbox upgrades, buildbot integration and BSP splitting come to mind. The minutes Leon sent today reflect that a well.

And we seriously need to consider renaming the TSC to Tentacle Steering Committee if we get caught in the yoctopus!

regards,

Koen

Re: OpenEmbedded and the Yocto Project

[Scott Garman]

On 11/02/2010 07:42 AM, Koen Kooi wrote:
> Before we go vote on stuff, I propose to try to work on relatively
> small focussed projects together to see how our goal and way of
> working match the yoctoproject ones. Things like tinderbox upgrades,
> buildbot integration and BSP splitting come to mind. The minutes Leon
> sent today reflect that a well.

Hello,

I am one of the folks who has been working on the Yocto Project (my work 
email is scott.a.garman@intel.com, IRC nick zenlinuxPDX). One of my 
primary responsibilities to date has been setting up and maintaining our 
Buildbot-based autobuilder.

http://autobuilder.pokylinux.org:8010

I am happy to lend a hand with Buildbot-related tasks and offer 
assistance as time permits.

Scott Garman

Re: OpenEmbedded and the Yocto Project

[Leon Woestenberg]

Hello,

On Tue, Nov 2, 2010 at 8:55 PM, Scott Garman <sgarman@zenlinux.com> wrote:
> On 11/02/2010 07:42 AM, Koen Kooi wrote:
>>
>> Before we go vote on stuff, I propose to try to work on relatively
>> small focussed projects together to see how our goal and way of
>> working match the yoctoproject ones. Things like tinderbox upgrades,
>> buildbot integration and BSP splitting come to mind. The minutes Leon
>> sent today reflect that a well.
>
Agreed. Creating an OpenEmbedded roadmap with such projects, so that
everybody sees what they can contribute to, would help.

We cannot really assign people (like sometimes a company can) but we
can ask them to do what they like, and make them like what they do
(mostly by being nice, oh, and visiting tropical islands once in a
while).

Typically, we have to spread our work more than a company does. I
think we did good over the last few years by attracting new developers
(despite the still steep learning curve). "But does it blend^Wscale?"

To extend a bit on the points still in my head from OEDEM:

- Tinderbox upgrades (automatically parse the collective build
database to extract useful information).
- Buildbot integration
- recipe tree namespace (BSP, ...) splitting in directories and/or overlays.
- automated tests and result reports on qemu targets
- automated tests and test reports on remote hardware board targets
- make available recipes from OpenEmbedded in Yocto(-OpenEmbedded)
- GCC 4.5.x
- merge Yocto/Poky improvements back to OpenEmbedded (checksum-tracked
progress from Richard's Poky proof-of-concept, cross-prelink)

> I am one of the folks who has been working on the Yocto Project (my work
> email is scott.a.garman@intel.com, IRC nick zenlinuxPDX). One of my primary
> responsibilities to date has been setting up and maintaining our
> Buildbot-based autobuilder.
>
Did you keep notes on how to setup such an autobuilder that you can
share in order to easily scale up this effort?

Regards,
-- 
Leon

Re: OpenEmbedded and the Yocto Project

[Scott Garman]

On 11/02/2010 03:00 PM, Leon Woestenberg wrote:
>> I am one of the folks who has been working on the Yocto Project (my work
>> email is scott.a.garman@intel.com, IRC nick zenlinuxPDX). One of my primary
>> responsibilities to date has been setting up and maintaining our
>> Buildbot-based autobuilder.
>>
> Did you keep notes on how to setup such an autobuilder that you can
> share in order to easily scale up this effort?

Hi Leon,

Actually, we even have a script to help automate setting up Buildbot:

http://git.pokylinux.org/cgit/cgit.cgi/poky-autobuilder/tree/scripts/poky-setup-autobuilder

It's been slightly sanitized and may require some minor tweaks. Also, 
we're planning on upgrading our Buildbot setup to 0.8.2 this month so 
there will be further updates coming soon. I will likely break this 
script up into master and slave setups, since Buildbot as of 0.8.1 
allows slave-only installations, which I imagine will be quite handy for 
OE testing volunteers.

In general, the poky-autobuilder repository has build and support 
scripts we're using (again, sometimes sanitized and in need of minor 
tweaks before you can run them yourself).

http://git.pokylinux.org/cgit/cgit.cgi/poky-autobuilder/

Scott

Re: OpenEmbedded and the Yocto Project

[Richard Purdie]

On Tue, 2010-11-02 at 15:42 +0100, Koen Kooi wrote:
> Op 2 nov 2010, om 15:16 heeft Philip Balister het volgende geschreven:
> 
> > The Linux Foundation has announced the Yocto Project (based partly 
> > on OpenEmbedded), http://yoctoproject.org/.
> > 
> > The OpenEmbedded eV board is pleased to see more people using 
> > OpenEmbedded and would like to develop a long term partnership with
> > the Yocto Project.
> > 
> > The eV board would like the OpenEmbedded Community to start 
> > discussing the Yocto Project and help develop a long term plan to
> > advance the goals of both projects. This process should result in a 
> > Voting Proposal that the OpenEmbedded eV can vote on.
> 
> Before we go vote on stuff, I propose to try to work on relatively
> small focussed projects together to see how our goal and way of
> working match the yoctoproject ones. Things like tinderbox upgrades,
> buildbot integration and BSP splitting come to mind. The minutes Leon
> sent today reflect that a well.

I'm more than happy to talk about what we're doing with Yocto, why and
what the differences are between Poky and OpenEmbedded. I covered a lot
of areas at OEDEM but I appreciate this didn't reach the people who
weren't there and I would like those people to understand what we're
doing too.

I'm not sure what the best forum is to have those discussions. I can
answer emails or we could do something more interactive on IRC or
similar, I'm open to ideas really...

Cheers,

Richard

Re: OpenEmbedded and the Yocto Project

[Khem Raj]

On Tue, Nov 2, 2010 at 3:24 PM, Richard Purdie <rpurdie@rpsys.net> wrote:
> On Tue, 2010-11-02 at 15:42 +0100, Koen Kooi wrote:
>> Op 2 nov 2010, om 15:16 heeft Philip Balister het volgende geschreven:
>>
>> > The Linux Foundation has announced the Yocto Project (based partly
>> > on OpenEmbedded), http://yoctoproject.org/.
>> >
>> > The OpenEmbedded eV board is pleased to see more people using
>> > OpenEmbedded and would like to develop a long term partnership with
>> > the Yocto Project.
>> >
>> > The eV board would like the OpenEmbedded Community to start
>> > discussing the Yocto Project and help develop a long term plan to
>> > advance the goals of both projects. This process should result in a
>> > Voting Proposal that the OpenEmbedded eV can vote on.
>>
>> Before we go vote on stuff, I propose to try to work on relatively
>> small focussed projects together to see how our goal and way of
>> working match the yoctoproject ones. Things like tinderbox upgrades,
>> buildbot integration and BSP splitting come to mind. The minutes Leon
>> sent today reflect that a well.
>
> I'm more than happy to talk about what we're doing with Yocto, why and
> what the differences are between Poky and OpenEmbedded. I covered a lot
> of areas at OEDEM but I appreciate this didn't reach the people who
> weren't there and I would like those people to understand what we're
> doing too.
>
> I'm not sure what the best forum is to have those discussions. I can
> answer emails or we could do something more interactive on IRC or
> similar, I'm open to ideas really...
>

host a webinar may be multiple times

> Cheers,
>
> Richard
>
>
>
>
>
>
> _______________________________________________
> Openembedded-members mailing list
> Openembedded-members@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-members
>

Re: OpenEmbedded and the Yocto Project

[Petr Štetiar]

Richard Purdie <rpurdie@rpsys.net> [2010-11-02 22:24:09]:

> I'm not sure what the best forum is to have those discussions. I can
> answer emails or we could do something more interactive on IRC or
> similar, I'm open to ideas really...

I would prefer this mailing list. It has threads, searchable archive etc.

-- ynezz

Re: OpenEmbedded and the Yocto Project

[Tian, Kevin]

>From: Philip Balister
>Sent: Sunday, November 07, 2010 5:01 AM
>
>On 11/06/2010 01:26 PM, Cliff Brake wrote:
>> On Fri, Nov 5, 2010 at 12:07 PM, Richard Purdie<rpurdie@rpsys.net>  wrote:
>>
>>> So what do I mean by hard? First there are the conversations with your
>>> legal department where some questions arise:
>>>
>>> * How do you ensure license compliance?
>>> * When someone makes a change, how do you know the license is still
>>>   correct?
>>> * You've mixing GPLv2 and v3? How are you ensuring no contamination?
>>> <cue a long discussion which I'd bet most companies have>
>>
>> Is GPLv3 the issue here with patent, DMC, and Tivoization issues?  Is
>> the solution to simply not have GPLv3 software in a device?
>>
>> Are there other license issues that companies are worried about?
>
>Cliff, I think each company has it's own licensing issues. The important
>thing for OE is to make sure our license fields are accurate.
>

Accurate and controllable so that you can easily come up a build which doesn't
contain sensitive licenses your customer are concerned and instead picks an
alternative serving same functionality if possible. :-)

Thanks
Kevin

Re: OpenEmbedded and the Yocto Project

[Philip Balister]

On 11/06/2010 01:26 PM, Cliff Brake wrote:
> On Fri, Nov 5, 2010 at 12:07 PM, Richard Purdie<rpurdie@rpsys.net>  wrote:
>
>> So what do I mean by hard? First there are the conversations with your
>> legal department where some questions arise:
>>
>> * How do you ensure license compliance?
>> * When someone makes a change, how do you know the license is still
>>   correct?
>> * You've mixing GPLv2 and v3? How are you ensuring no contamination?
>> <cue a long discussion which I'd bet most companies have>
>
> Is GPLv3 the issue here with patent, DMC, and Tivoization issues?  Is
> the solution to simply not have GPLv3 software in a device?
>
> Are there other license issues that companies are worried about?

Cliff, I think each company has it's own licensing issues. The important 
thing for OE is to make sure our license fields are accurate.

Philip

Re: OpenEmbedded and the Yocto Project

[Stefan Schmidt]

Hello.

On Sat, 2010-11-06 at 13:26, Cliff Brake wrote:
> On Fri, Nov 5, 2010 at 12:07 PM, Richard Purdie <rpurdie@rpsys.net> wrote:
> 
> > So what do I mean by hard? First there are the conversations with your
> > legal department where some questions arise:
> >
> > * How do you ensure license compliance?
> > * When someone makes a change, how do you know the license is still
> >  correct?
> > * You've mixing GPLv2 and v3? How are you ensuring no contamination?
> > <cue a long discussion which I'd bet most companies have>
> 
> Is GPLv3 the issue here with patent, DMC, and Tivoization issues?  Is
> the solution to simply not have GPLv3 software in a device?

At least that is what I hear most of the times. Yocto should be able to build a
GPLv3-free image. Obviously this kinda results in keeping old versions of core
elements of software maintained by the FSF.

> Are there other license issues that companies are worried about?

Well, we have Google who demostrated that they would rather write everything
from scratch, including a libc, instead of using the GPL. :)

But that is not a problem for OE. All other players have realized how to deal
with the GPL.

regards
Stefan Schmidt

Re: OpenEmbedded and the Yocto Project

[Cliff Brake]

On Fri, Nov 5, 2010 at 12:07 PM, Richard Purdie <rpurdie@rpsys.net> wrote:

> So what do I mean by hard? First there are the conversations with your
> legal department where some questions arise:
>
> * How do you ensure license compliance?
> * When someone makes a change, how do you know the license is still
>  correct?
> * You've mixing GPLv2 and v3? How are you ensuring no contamination?
> <cue a long discussion which I'd bet most companies have>

Is GPLv3 the issue here with patent, DMC, and Tivoization issues?  Is
the solution to simply not have GPLv3 software in a device?

Are there other license issues that companies are worried about?

Thanks,
Cliff

-- 
=================
http://bec-systems.com

Re: OpenEmbedded and the Yocto Project

[Frans Meulenbroeks]

2010/11/5 Richard Purdie <rpurdie@rpsys.net>:
> On Fri, 2010-11-05 at 17:36 +0100, Paul Menzel wrote:
>> Am Freitag, den 05.11.2010, 16:07 +0000 schrieb Richard Purdie:
>> > Hopefully this email explains some of the reasons the project exists,
>> > what the differences are to OE and that is isn't something bad but
>> > actually a very positive thing. I have not dived into the topic of
>> > if/how Poky and OE can work together, that can be the subject of another
>> > email. If anyone has any questions I'm happy to answer them.
>>
>> Actually this is in my opinion the most important question. In the
>> beginning you talk about the one good build system.
>>
>> But in the end to get the one good build system, OpenEmbedded should
>> somehow be merged into or replaced by Poky(?) or be replaced, should not
>> it?
>>
>> I have only some small experience with and used only OpenEmbedded. So I
>> do not know about Poky and the other build systems. But I agree with you
>> to try to avoid duplicate work.
>>
>> So a message about your vision about the future of OE would be greatly
>> appreciated.
>
> The thing to keep in mind is that Poky and OE are very similar, sharing
> bitbake, the recipe format and pretty much the same class files.
>
> We therefore we didn't start a new build system project we took an
> existing one very similar to OpenEmbedded and we plan to keep the
> architecture the same. Even since Poky was created over five years ago
> I've been publicly discussing new features, how they're best implemented
> and so on with OE developers, bitbake maintainers and the OE TSC so the
> projects stay roughly in sync.
>
> I've stated what Yocto/Poky is doing differently and why, the question
> now is whether the OE developers agree/disagree with those things and if
> they are prepared to change OE at all to address some of the reasons
> Yocto is doing some things differently.
>
> There are ways the two projects can complement each other but there has
> to be a will there to do it. Traditionally, OE has trouble making
> decisions which I think it hurts the project in general. I'm hoping this
> topic isn't going to be one of those things it fails to reach a
> conclusion on.
>

Well, I for me I have some reservations, but I definitely also see
plenty of options as yocto does address several issues that
historically were not among the strong points of OE.
Let's work on combining the best of both worlds!

Frans

Re: OpenEmbedded and the Yocto Project

[Richard Purdie]

On Fri, 2010-11-05 at 17:36 +0100, Paul Menzel wrote:
> Am Freitag, den 05.11.2010, 16:07 +0000 schrieb Richard Purdie:
> > Hopefully this email explains some of the reasons the project exists,
> > what the differences are to OE and that is isn't something bad but
> > actually a very positive thing. I have not dived into the topic of
> > if/how Poky and OE can work together, that can be the subject of another
> > email. If anyone has any questions I'm happy to answer them.
> 
> Actually this is in my opinion the most important question. In the
> beginning you talk about the one good build system.
> 
> But in the end to get the one good build system, OpenEmbedded should
> somehow be merged into or replaced by Poky(?) or be replaced, should not
> it?
> 
> I have only some small experience with and used only OpenEmbedded. So I
> do not know about Poky and the other build systems. But I agree with you
> to try to avoid duplicate work.
> 
> So a message about your vision about the future of OE would be greatly
> appreciated.

The thing to keep in mind is that Poky and OE are very similar, sharing
bitbake, the recipe format and pretty much the same class files.

We therefore we didn't start a new build system project we took an
existing one very similar to OpenEmbedded and we plan to keep the
architecture the same. Even since Poky was created over five years ago
I've been publicly discussing new features, how they're best implemented
and so on with OE developers, bitbake maintainers and the OE TSC so the
projects stay roughly in sync.

I've stated what Yocto/Poky is doing differently and why, the question
now is whether the OE developers agree/disagree with those things and if
they are prepared to change OE at all to address some of the reasons
Yocto is doing some things differently.

There are ways the two projects can complement each other but there has
to be a will there to do it. Traditionally, OE has trouble making
decisions which I think it hurts the project in general. I'm hoping this
topic isn't going to be one of those things it fails to reach a
conclusion on.

Cheers,

Richard

Re: OpenEmbedded and the Yocto Project

[Chris Larson]

On Fri, Nov 5, 2010 at 9:07 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
>
> When it comes to using OE commercially, its actually quite hard. If you
> look at even the big companies using OE like TI, Mentor and Montavista
> to name but a few, whilst they are using it, there are significant
> changes they need to make to turn it into a commercial product. Those
> changes and stabilisation are eating a lot of the developer resources
> they put in and there is arguably a comparatively low amount of feature
> development being worked upon. If somehow we could share some of that
> effort, it would free up resources for feature development.
>

I think we can all agree with that.


> End result is that Yocto/Poky have a states release cycle, approximately
> six months. We have releases, BSPs can be made against the last release.
> When planning people can know roughly what state the tree will be in at
> a given time and whilst no timeline is perfect for everyone, its well
> defined and people can adapt to it.
>

This is good to hear, and I hope OE can adopt this as well, or,
alternatively, as has been discussed a bit, OE could become a set of layers
built upon Yocto, and share the release cycle.

Third is a QA problem and one of size and scale. Realistically, there is
> no way you can "test" OE as its just too large with too many
> combinations. On the other hand we have a pretty good known test matrix
> for Poky/Yocto as its a smaller set of data than OE. If we want to
> expand that set, we're prepared to as long as there is contribution to
> help test it too (additional autobuilder machines and QA resource).
>
> We do appreciate the value of the additional things OE contains however
> and also realise the value in additional machine support. This is why
> the Yocto project has done work to make layers work better and become
> usable for additional topic "metadata" layers and also machine support
> in the form of BSPs. This neatly allows people to take ownership of
> subsets of the metadata, be them community or commercial, a problem OE
> has long suffered from.
>

This is also good to hear, and I agree that this is a good solution -- we
actually always wanted to be able to split control and ownership in this
way, but originally determining how to combine metadata from multiple places
wasn't a priority.  Aside: the approach taken by Conary for the metadata is
a very intriguing one.

Having a known test matrix and an ability to QA the system also helps
> massively with feature development. Within Poky/Yocto we can *know*
> whether a given change causes a regression within the core without too
> much effort. I'd suggest this is much much harder to know within OE.
>

Perhaps, but I think the OE Testing effort is a step in the right direction,
so OE too will have a test matrix, just, as you say, a less complete one.


> This brings me to a forth problem which is related to some of the above
> indirectly which is that of the contributions model itself. Above, I
> mentioned we needed to ensure some kind of vetting of the license on new
> recipes and this isn't possible with the OE "push" contributions model.
> This is why Yocto/Poky have adopted a kernel like "pull" model for
> changes. If it can scale for the Linux kernel, I think it can scale for
> Poky/Yocto and could if there was a will, for OE too. Sub-maintainers
> are appointed based on trust to merge together "pull" requests and pass
> them upwards so no one person becomes overloaded.
>
> This "pull" model also makes decisions and technical direction easier,
> there has to be a lot of trust in the Linus type person to do the right
> thing. If that person screws up, the project can fork and there would
> likely be a clear winner afterwards if the decision was worth the fork
> so there is actually a strong natural pressure on that person to get
> things right. In the Yocto/Poky world, I've taken on that overall
> responsibility as I have relevant technical experience and a long
> history of leading Poky and to a large extent OE from a technical
> standpoint having contributed many of the technical advancements and
> architecture improvements that have put OE where it is today. I fully
> intend to have sub maintainers to help and I really care about their
> technical leadership, not who pays their wages.
>
> This is as good a time as any to talk about the governance of the Yocto
> Project as people look at the diagram on the website and get very
> confused. The people with the power in the model are the architect/
> maintainer groups.
>
> When it comes to technical direction this comes from the maintainers of
> the various components. They set this based on the input from the
> architect/steering group but they are the ones deciding the
> implementation details, writing code, accepting patches and developing
> the feature roadmaps and development schedules and actually doing the
> real work.
>
> One of the cornerstones of the governance is that if the steering group
> ever has to "tell" the architect what to do then the architect has
> failed at their job. This also applies to maintainers as if the
> architect has to get directly involved things aren't working correctly.
>
> So the steering group is there to suggest things that would
> help the project and steer to a degree but the architect is there to
> actually set the technical direction overall. In turn the architect is
> there to given direction to the maintainers.
>
> The real power therefore rests with the Maintainers with delegated
> authority through the Architect. The steering group gives managers at
> the companies involved a warm fuzzy feeling that they can get involved
> if things ever do look like they're going of the rails but the agreement
> is its "hands off" as much as possible. Its also worth noting the CTOs
> on the diagram are primarily about publicising and networking the
> project.
>

I really like this approach, personally.  I think OE has lost direction
somewhat since I left the project, even as it has grown in leaps and bounds,
not because of me, but because of the reduction of technical leadership in
general.  Having both working groups / steering committees to give companies
input and individual technical leaders to do day to day decision making
seems to be a good solution.  I do think we'll need a branch structure which
facilitates integration and stability testing of features before they get
pulled, along the lines of linux-next or git's next and pu branches, though,
if it doesn't have that already :)
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: OpenEmbedded and the Yocto Project

[Paul Menzel]

[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]

Am Freitag, den 05.11.2010, 16:07 +0000 schrieb Richard Purdie:
> I've talked to various people and I think the point has come to write
> down my thoughts rather than repeat conversations with various people.
> Sorry if this ends up as a long email.

Thank you for writing this. I am aware that this probably took you quite
some time.

> I'm on record in the past for commenting on the number of build system
> presentations at ELC-E in 2009. The embedded community has several of
> them, all with advantages and disadvantages, I look at this and wonder
> why we couldn't have one good one. You also wonder why there is no focus
> point fostering embedded Linux tools in general, everyone loves to go
> off and do their own thing as its easier than working together.
> 
> It was based on these observations and a desire to improve the
> situation, I proposed the idea of what has become the Yocto project.

[…]

> This project is not in competition to OpenEmbedded, its a compliment to
> it. Its a serious opportunity to leverage some corporate sponsorship and
> see OpenEmbedded, the Yocto project and also Embedded Linux all grow.

[…]

> Hopefully this email explains some of the reasons the project exists,
> what the differences are to OE and that is isn't something bad but
> actually a very positive thing. I have not dived into the topic of
> if/how Poky and OE can work together, that can be the subject of another
> email. If anyone has any questions I'm happy to answer them.

Actually this is in my opinion the most important question. In the
beginning you talk about the one good build system.

But in the end to get the one good build system, OpenEmbedded should
somehow be merged into or replaced by Poky(?) or be replaced, should not
it?

I have only some small experience with and used only OpenEmbedded. So I
do not know about Poky and the other build systems. But I agree with you
to try to avoid duplicate work.

So a message about your vision about the future of OE would be greatly
appreciated.


Thanks,

Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

OpenEmbedded and the Yocto Project

[Richard Purdie]

I've talked to various people and I think the point has come to write
down my thoughts rather than repeat conversations with various people.
Sorry if this ends up as a long email.

I'm on record in the past for commenting on the number of build system
presentations at ELC-E in 2009. The embedded community has several of
them, all with advantages and disadvantages, I look at this and wonder
why we couldn't have one good one. You also wonder why there is no focus
point fostering embedded Linux tools in general, everyone loves to go
off and do their own thing as its easier than working together.

It was based on these observations and a desire to improve the
situation, I proposed the idea of what has become the Yocto project.

Intel doesn't want to own Yocto, it realises that to create this one
good build system for example, it needs other people to be involved.
This is why the Linux Foundation now owns the rights to Poky and the
Yocto Project. Yes, it been seeded by Intel/Wind River people and by
Poky, not OpenEmbedded. The simple reason is that these are the things
could be donated to the project, OpenEmbedded is not something the Linux
Foundation can take ownership of, nor should it be.

There are many interesting facets to the Yocto project and I will cover
some of them below later as I don't think people are seeing them yet but
I know what most readers are interested in here so getting to the heart
of the matter, OpenEmbedded and Poky.

OpenEmbedded has always intended itself to be useful to companies to
build products off and to become an all purpose fully featured build
system and development tool. It does many things well,  I love it for
what it is and I've spent over half a decade working on it in various
ways which I'd never have done that if I didn't believe in the
architecture.

When it comes to using OE commercially, its actually quite hard. If you
look at even the big companies using OE like TI, Mentor and Montavista
to name but a few, whilst they are using it, there are significant
changes they need to make to turn it into a commercial product. Those
changes and stabilisation are eating a lot of the developer resources
they put in and there is arguably a comparatively low amount of feature
development being worked upon. If somehow we could share some of that
effort, it would free up resources for feature development.

So what do I mean by hard? First there are the conversations with your
legal department where some questions arise:

* How do you ensure license compliance?
* When someone makes a change, how do you know the license is still 
  correct?
* You've mixing GPLv2 and v3? How are you ensuring no contamination?
<cue a long discussion which I'd bet most companies have>

Bottom line is that the contributions process to OE is not sufficient to
be acceptable to most companies legal departments. Sure, instead of
Yocto, Intel and Wind River could each have some internal version of OE
which we'd vet changes to and I suspect Montavista and Mentor at least
already have this but its not very open source and it duplicates a lot
of work.

So the end result is that the Yocto project has totally vetted the
LICENSE data in Poky, any new additions go through a set of checks
before they be allowed into the repository and we also have some
automated process/tools enhancements such as the licence checksums to
catch changes to licenses more easily.

So that is one area, second is one of development and releases. OE has
never had one, it may have one soon, we'll see but so far its never
happened. Releases are important in many ways as they give people
something to plan expectations against. Currently OE is always active
development, there is never any calm down before a release, switch to
bug fixing mode rather than enhancement and so forth. This means if you
develop a BSP layer, which version of OE can you state it works with?

End result is that Yocto/Poky have a states release cycle, approximately
six months. We have releases, BSPs can be made against the last release.
When planning people can know roughly what state the tree will be in at
a given time and whilst no timeline is perfect for everyone, its well
defined and people can adapt to it.

Third is a QA problem and one of size and scale. Realistically, there is
no way you can "test" OE as its just too large with too many
combinations. On the other hand we have a pretty good known test matrix
for Poky/Yocto as its a smaller set of data than OE. If we want to
expand that set, we're prepared to as long as there is contribution to
help test it too (additional autobuilder machines and QA resource). 

We do appreciate the value of the additional things OE contains however
and also realise the value in additional machine support. This is why
the Yocto project has done work to make layers work better and become
usable for additional topic "metadata" layers and also machine support
in the form of BSPs. This neatly allows people to take ownership of
subsets of the metadata, be them community or commercial, a problem OE
has long suffered from.

Having a known test matrix and an ability to QA the system also helps
massively with feature development. Within Poky/Yocto we can *know*
whether a given change causes a regression within the core without too
much effort. I'd suggest this is much much harder to know within OE.

This brings me to a forth problem which is related to some of the above
indirectly which is that of the contributions model itself. Above, I
mentioned we needed to ensure some kind of vetting of the license on new
recipes and this isn't possible with the OE "push" contributions model.
This is why Yocto/Poky have adopted a kernel like "pull" model for
changes. If it can scale for the Linux kernel, I think it can scale for
Poky/Yocto and could if there was a will, for OE too. Sub-maintainers
are appointed based on trust to merge together "pull" requests and pass
them upwards so no one person becomes overloaded.

This "pull" model also makes decisions and technical direction easier,
there has to be a lot of trust in the Linus type person to do the right
thing. If that person screws up, the project can fork and there would
likely be a clear winner afterwards if the decision was worth the fork
so there is actually a strong natural pressure on that person to get
things right. In the Yocto/Poky world, I've taken on that overall
responsibility as I have relevant technical experience and a long
history of leading Poky and to a large extent OE from a technical
standpoint having contributed many of the technical advancements and
architecture improvements that have put OE where it is today. I fully
intend to have sub maintainers to help and I really care about their
technical leadership, not who pays their wages.

This is as good a time as any to talk about the governance of the Yocto
Project as people look at the diagram on the website and get very
confused. The people with the power in the model are the architect/
maintainer groups.

When it comes to technical direction this comes from the maintainers of
the various components. They set this based on the input from the
architect/steering group but they are the ones deciding the
implementation details, writing code, accepting patches and developing
the feature roadmaps and development schedules and actually doing the
real work.

One of the cornerstones of the governance is that if the steering group
ever has to "tell" the architect what to do then the architect has
failed at their job. This also applies to maintainers as if the
architect has to get directly involved things aren't working correctly.

So the steering group is there to suggest things that would
help the project and steer to a degree but the architect is there to
actually set the technical direction overall. In turn the architect is
there to given direction to the maintainers.

The real power therefore rests with the Maintainers with delegated
authority through the Architect. The steering group gives managers at
the companies involved a warm fuzzy feeling that they can get involved
if things ever do look like they're going of the rails but the agreement
is its "hands off" as much as possible. Its also worth noting the CTOs
on the diagram are primarily about publicising and networking the
project.

I, Intel, Wind River and the Linux Foundation really hope we're going to
see more people becoming part of the Yocto project, particularly
companies at the steering level and hence new maintainers (the people
who get things done ;-).

This project is not in competition to OpenEmbedded, its a compliment to
it. Its a serious opportunity to leverage some corporate sponsorship and
see OpenEmbedded, the Yocto project and also Embedded Linux all grow.

In this email I've talked a lot about the build system but there is much
more to this than just that as our focus is wider. We're interested in
embedded tools in general for example:

* Yocto's Eclipse/Anjuta Plugins
* Pseduo (fakeroot replacement)
* BSP standardisation
* Cross Prelink
* Documentation
* Hardware labs/testing framework

to name just a few things we're talked about. There are many others and
many interesting developments yet to come including the things we've not
yet through of but new members will bring!

Hopefully this email explains some of the reasons the project exists,
what the differences are to OE and that is isn't something bad but
actually a very positive thing. I have not dived into the topic of
if/how Poky and OE can work together, that can be the subject of another
email. If anyone has any questions I'm happy to answer them.

Cheers,

Richard