A new bitbake extension: .bbappend files

A new bitbake extension: .bbappend files

[Richard Purdie]

Whilst our layers mechanism, is great it does have a drawback which has
bugged me for a while. If you have a recipe like pointercal which has
machine specific information in it and you have your new machine code in
a layer, how do you add a pointercal file for your machine?

Answer is you copy the whole pointercal recipe and files into your
layer, then add the single file for your machine. To me this is ugly,
ugly, ugly. We hate code duplication and as soon as you create two
copies of the same information, we've failed.

So how could we do this better? Somehow we need to say that a given
directory X has some information which should be merged with the
original recipe. I've thought through several different ways of doing
this and the best solution I found was "bbappend".

The idea is that if bitbake finds any X.bbappend files, when it loads
X.bb, it will also include these files after it parses the base .bb file
(but before finalise and the anonymous methods run). This means that
the .bbappend file can poke around and do whatever it might want to the
recipe to customise it.

I went ahead and tried it out as its quite simple to code this in
bitbake. I liked the result enough I've already merged this into Poky:

http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660

I'm proposing to push it to bitbake master if there are no serious
objections.

As an example use case, for the pointercal case above in another later
you could add a pointercal_0.0.bbappend file which contained something
like:

FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d, True))}"

which would then cause the directory containing the bbappend file to be
searched for pointercal files.

There are of course many other uses this could be put to for creating
customised layers, its totally generic.

For the specific case of paths, I have wondered if there would be a way
to leverage help from bitbake in creating a sane set of search paths but
I'm still thinking about that. This extension is good enough in its own
right in my opinion to be worthwhile.

Cheers,

Richard

Re: A new bitbake extension: .bbappend files

[George C. Huntington, III]

I love this idea.  seeing this would go a long way towards getting my
employers on board with using openembedded and even getting to contribute.

Re: A new bitbake extension: .bbappend files

[Carl Simonson]

I like it too.

So does this mean that PR bumps happen in the bbappend file? Obviously
you need to have the recipe rebuilt if a change is made.

Carl Simonson
simonsonc@gmail.com



On Fri, Jul 16, 2010 at 9:35 AM, George C. Huntington, III
<george.huntingtoniii@gmail.com> wrote:
> I love this idea.  seeing this would go a long way towards getting my
> employers on board with using openembedded and even getting to contribute.
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Michael Smith]

Hi Richard,

Does this do anything different than amendrecipes.bbclass?

Mike

Richard Purdie wrote:
> The idea is that if bitbake finds any X.bbappend files, when it loads
> X.bb, it will also include these files after it parses the base .bb file
> (but before finalise and the anonymous methods run). This means that
> the .bbappend file can poke around and do whatever it might want to the
> recipe to customise it.

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Chris Larson]

It basically solves the same problem.  It's slightly less flexible, in that
it has to apply to a specific .bb, whereas an amend.inc can apply to any
version of a recipe, etc, depending on where it resides in the FILESPATH,
but amend.inc cannot be integrated into bitbake without a performance
impact, because we don't necessarily know the FILESPATH until after the
finalizing steps at the end of the parsing process, and then it has to
re-finalize after the amend.inc is parsed, to ensure its anonymous python
functions are executed.  The advantage to this is its simplicity in
implementation and the miniscule performance impact.  One slight concern is
its assumption that the basename of the filename is unique, but I expect
that's okay in most real world cases.

I hope to spend some time poking at this today.

-Chris

On Fri, Jul 16, 2010 at 8:46 AM, Michael Smith <msmith@cbnco.com> wrote:

> Hi Richard,
>
> Does this do anything different than amendrecipes.bbclass?
>
> Mike
>
>
> Richard Purdie wrote:
>
>> The idea is that if bitbake finds any X.bbappend files, when it loads
>> X.bb, it will also include these files after it parses the base .bb file
>> (but before finalise and the anonymous methods run). This means that
>> the .bbappend file can poke around and do whatever it might want to the
>> recipe to customise it.
>>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>



-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Richard Purdie]

On Fri, 2010-07-16 at 11:46 -0400, Michael Smith wrote:
> Richard Purdie wrote:
> > The idea is that if bitbake finds any X.bbappend files, when it loads
> > X.bb, it will also include these files after it parses the base .bb file
> > (but before finalise and the anonymous methods run). This means that
> > the .bbappend file can poke around and do whatever it might want to the
> > recipe to customise it.
> 
> Does this do anything different than amendrecipes.bbclass?

Several people have mentioned the amend bbclass. This implements
something very similar in quite a different way as part of the bitbake
tool itself. There are advantages and disadvantages to doing this.

The features I liked about the bbappend approach was the small amount of
code change required to bitbake compared to the functionality it offers
and also the overhead of the bbappend code is minimal.

I also feel the bbappend format is slightly easier for users to
comprehend but that is subjective.

A downside is that the .bbappend applies to specific versions, not all
versions as amend.bbclass can be setup. There are ways that could be
changed.

The amend class has a greater degree of flexibility in that you can make
changes to the class and it could be argued not changing the core is an
advantage too.

Cheers,

Richard

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Frans Meulenbroeks]

2010/7/16 Richard Purdie <rpurdie@rpsys.net>

> On Fri, 2010-07-16 at 11:46 -0400, Michael Smith wrote:
> > Richard Purdie wrote:
> > > The idea is that if bitbake finds any X.bbappend files, when it loads
> > > X.bb, it will also include these files after it parses the base .bb
> file
> > > (but before finalise and the anonymous methods run). This means that
> > > the .bbappend file can poke around and do whatever it might want to the
> > > recipe to customise it.
> >
> > Does this do anything different than amendrecipes.bbclass?
>
> Several people have mentioned the amend bbclass. This implements
> something very similar in quite a different way as part of the bitbake
> tool itself. There are advantages and disadvantages to doing this.
>
> The features I liked about the bbappend approach was the small amount of
> code change required to bitbake compared to the functionality it offers
> and also the overhead of the bbappend code is minimal.
>
> I also feel the bbappend format is slightly easier for users to
> comprehend but that is subjective.
>
> A downside is that the .bbappend applies to specific versions, not all
> versions as amend.bbclass can be setup. There are ways that could be
> changed.
>
> The amend class has a greater degree of flexibility in that you can make
> changes to the class and it could be argued not changing the core is an
> advantage too.
>
> Cheers,
>
> Richard,

Looks like a fine idea to me (although I must say I was unaware of the amend
class, so I cannot really compare).

An alternative I was thinking of was to have a mechanism in an overlayed
recipe that would allow you to include whatever version was in a lower
priority overlay, but your prososal looks also quite sound.

+1

Frans

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Esben Haabendal]

On Fri, Jul 16, 2010 at 6:53 PM, Frans Meulenbroeks
<fransmeulenbroeks@gmail.com> wrote:
> Looks like a fine idea to me (although I must say I was unaware of the amend
> class, so I cannot really compare).
>
> An alternative I was thinking of was to have a mechanism in an overlayed
> recipe that would allow you to include whatever version was in a lower
> priority overlay, but your prososal looks also quite sound.

I have been thinking of this approach several times also.
Something like

overlay_require pointercal_0.0.bb

But I like the bbappend approach also, so

+1

from me also.  Simple is good :-)

/Esben

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Detlef Vollmann]

On 07/16/10 16:25, Richard Purdie wrote:
> For the specific case of paths, I have wondered if there would be a way
> to leverage help from bitbake in creating a sane set of search paths but
> I'm still thinking about that. This extension is good enough in its own
> right in my opinion to be worthwhile.
This might be true, but solving the general path problem would be
much more useful.
If you want to add a new version of e.g. gcc in your ovlerlay,
you have to pull over the whole gcc directory with all its .inc classes.
This can't be solved by .bbapend, or am I missing something?

Regards,
   Detlef

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Chris Larson]

On Sun, Jul 18, 2010 at 3:06 AM, Detlef Vollmann <dv@vollmann.ch> wrote:

> On 07/16/10 16:25, Richard Purdie wrote:
>
>> For the specific case of paths, I have wondered if there would be a way
>> to leverage help from bitbake in creating a sane set of search paths but
>> I'm still thinking about that. This extension is good enough in its own
>> right in my opinion to be worthwhile.
>>
> This might be true, but solving the general path problem would be
> much more useful.
> If you want to add a new version of e.g. gcc in your ovlerlay,
> you have to pull over the whole gcc directory with all its .inc classes.
> This can't be solved by .bbapend, or am I missing something?


You can handle that today by adding to FILESPATHBASE.  All file:// files
from SRC_URI are located using FILESPATH, which is constructed from
FILESPATHBASE, FILESPATHPKG, and OVERRIDES.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Detlef Vollmann]

On 07/18/10 17:13, Chris Larson wrote:
> On Sun, Jul 18, 2010 at 3:06 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
> 
>> On 07/16/10 16:25, Richard Purdie wrote:
>>
>>> For the specific case of paths, I have wondered if there would be a way
>>> to leverage help from bitbake in creating a sane set of search paths but
>>> I'm still thinking about that. This extension is good enough in its own
>>> right in my opinion to be worthwhile.
>>>
>> This might be true, but solving the general path problem would be
>> much more useful.
>> If you want to add a new version of e.g. gcc in your ovlerlay,
>> you have to pull over the whole gcc directory with all its .inc classes.
>> This can't be solved by .bbapend, or am I missing something?
> 
> 
> You can handle that today by adding to FILESPATHBASE.  All file:// files
> from SRC_URI are located using FILESPATH, which is constructed from
> FILESPATHBASE, FILESPATHPKG, and OVERRIDES.
AFAIK this doen't work for include/require :-(

   Detlef

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Chris Larson]

On Sun, Jul 18, 2010 at 9:01 AM, Detlef Vollmann <dv@vollmann.ch> wrote:

> On 07/18/10 17:13, Chris Larson wrote:
>
>> On Sun, Jul 18, 2010 at 3:06 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
>>
>>  On 07/16/10 16:25, Richard Purdie wrote:
>>>
>>>  For the specific case of paths, I have wondered if there would be a way
>>>> to leverage help from bitbake in creating a sane set of search paths but
>>>> I'm still thinking about that. This extension is good enough in its own
>>>> right in my opinion to be worthwhile.
>>>>
>>>>  This might be true, but solving the general path problem would be
>>> much more useful.
>>> If you want to add a new version of e.g. gcc in your ovlerlay,
>>> you have to pull over the whole gcc directory with all its .inc classes.
>>> This can't be solved by .bbapend, or am I missing something?
>>>
>>
>>
>> You can handle that today by adding to FILESPATHBASE.  All file:// files
>> from SRC_URI are located using FILESPATH, which is constructed from
>> FILESPATHBASE, FILESPATHPKG, and OVERRIDES.
>>
> AFAIK this doen't work for include/require :-(


Don't copy the recipe at all, use .bbappend instead, and you won't need to
have the .inc in a different collection from the recipe itself.

Alternatively, you can adjust BBPATH to also include the other collection
based paths + os.path.dirname('${FILE_DIRNAME}') or so.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Detlef Vollmann]

On 07/18/10 18:06, Chris Larson wrote:
> On Sun, Jul 18, 2010 at 9:01 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
> 
>> On 07/18/10 17:13, Chris Larson wrote:
>>
>>> On Sun, Jul 18, 2010 at 3:06 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
>>>
>>>  On 07/16/10 16:25, Richard Purdie wrote:
>>>>  For the specific case of paths, I have wondered if there would be a way
>>>>> to leverage help from bitbake in creating a sane set of search paths but
>>>>> I'm still thinking about that. This extension is good enough in its own
>>>>> right in my opinion to be worthwhile.
>>>>>
>>>>>  This might be true, but solving the general path problem would be
>>>> much more useful.
>>>> If you want to add a new version of e.g. gcc in your ovlerlay,
>>>> you have to pull over the whole gcc directory with all its .inc classes.
>>>> This can't be solved by .bbapend, or am I missing something?
>>>>
>>>
>>> You can handle that today by adding to FILESPATHBASE.  All file:// files
>>> from SRC_URI are located using FILESPATH, which is constructed from
>>> FILESPATHBASE, FILESPATHPKG, and OVERRIDES.
>>>
>> AFAIK this doen't work for include/require :-(
> 
> 
> Don't copy the recipe at all, use .bbappend instead, and you won't need to
> have the .inc in a different collection from the recipe itself.
In that case I didn't understand .bbappend.
If I have a recipe gcc_4.3.2.bb in e.g. oe-stable/recipes/gcc, but in
that directory is no gcc_4.4.4.bb,  and I put gcc_4.4.4.bbappend into
myproject/recipes/gcc, that would probably given an error like
'no .bb file to append'.
At least that's how I understood .bbappend.

> Alternatively, you can adjust BBPATH to also include the other collection
> based paths + os.path.dirname('${FILE_DIRNAME}') or so.
I'd be very interested to find a way so that a 'require gcc.inc' in
myproject/recipes/gcc_4.4.4.bb would find oe-stable/recipes/gcc/gcc.inc.
So far I failed.

   Detlef

Re: A new bitbake extension: .bbappend files

[Frans Meulenbroeks]

2010/7/16 Richard Purdie <rpurdie@rpsys.net>

> Whilst our layers mechanism, is great it does have a drawback which has
> bugged me for a while. If you have a recipe like pointercal which has
> machine specific information in it and you have your new machine code in
> a layer, how do you add a pointercal file for your machine?
>
> Answer is you copy the whole pointercal recipe and files into your
> layer, then add the single file for your machine. To me this is ugly,
> ugly, ugly. We hate code duplication and as soon as you create two
> copies of the same information, we've failed.
>
> So how could we do this better? Somehow we need to say that a given
> directory X has some information which should be merged with the
> original recipe. I've thought through several different ways of doing
> this and the best solution I found was "bbappend".
>
> The idea is that if bitbake finds any X.bbappend files, when it loads
> X.bb, it will also include these files after it parses the base .bb file
> (but before finalise and the anonymous methods run). This means that
> the .bbappend file can poke around and do whatever it might want to the
> recipe to customise it.
>
> I went ahead and tried it out as its quite simple to code this in
> bitbake. I liked the result enough I've already merged this into Poky:
>
>
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
>
> I'm proposing to push it to bitbake master if there are no serious
> objections.
>
> As an example use case, for the pointercal case above in another later
> you could add a pointercal_0.0.bbappend file which contained something
> like:
>
> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d,
> True))}"
>
> which would then cause the directory containing the bbappend file to be
> searched for pointercal files.
>
> There are of course many other uses this could be put to for creating
> customised layers, its totally generic.
>
> For the specific case of paths, I have wondered if there would be a way
> to leverage help from bitbake in creating a sane set of search paths but
> I'm still thinking about that. This extension is good enough in its own
> right in my opinion to be worthwhile.
>
> Cheers,
>
> Richard
>

I've been pondering a litlte bit more about this.
Will the .bbappend mechanism allow me to override a file in the original
recipe?
If so how?

Typical use case: I want to have a dedicated defconfig for my linux kernel.
Alternate use case: a dedicated busybox configuration.

Frans.

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Richard Purdie]

On Sun, 2010-07-18 at 21:24 +0200, Detlef Vollmann wrote:
> On 07/18/10 18:06, Chris Larson wrote:
> > On Sun, Jul 18, 2010 at 9:01 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
> > 
> >> On 07/18/10 17:13, Chris Larson wrote:
> >>
> >>> On Sun, Jul 18, 2010 at 3:06 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
> >>>
> >>>  On 07/16/10 16:25, Richard Purdie wrote:
> >>>>  For the specific case of paths, I have wondered if there would be a way
> >>>>> to leverage help from bitbake in creating a sane set of search paths but
> >>>>> I'm still thinking about that. This extension is good enough in its own
> >>>>> right in my opinion to be worthwhile.
> >>>>>
> >>>>>  This might be true, but solving the general path problem would be
> >>>> much more useful.
> >>>> If you want to add a new version of e.g. gcc in your ovlerlay,
> >>>> you have to pull over the whole gcc directory with all its .inc classes.
> >>>> This can't be solved by .bbapend, or am I missing something?
> >>>>
> >>>
> >>> You can handle that today by adding to FILESPATHBASE.  All file:// files
> >>> from SRC_URI are located using FILESPATH, which is constructed from
> >>> FILESPATHBASE, FILESPATHPKG, and OVERRIDES.
> >>>
> >> AFAIK this doen't work for include/require :-(
> > 
> > 
> > Don't copy the recipe at all, use .bbappend instead, and you won't need to
> > have the .inc in a different collection from the recipe itself.
> In that case I didn't understand .bbappend.
> If I have a recipe gcc_4.3.2.bb in e.g. oe-stable/recipes/gcc, but in
> that directory is no gcc_4.4.4.bb,  and I put gcc_4.4.4.bbappend into
> myproject/recipes/gcc, that would probably given an error like
> 'no .bb file to append'.
> At least that's how I understood .bbappend.
> 
> > Alternatively, you can adjust BBPATH to also include the other collection
> > based paths + os.path.dirname('${FILE_DIRNAME}') or so.
> I'd be very interested to find a way so that a 'require gcc.inc' in
> myproject/recipes/gcc_4.4.4.bb would find oe-stable/recipes/gcc/gcc.inc.
> So far I failed.

Does something like "require recipes/gcc/gcc.inc" not just work already
as it will search BBPATH...

Cheers,

Richard

Re: A new bitbake extension: .bbappend files

[Chris Larson]

On Sun, Jul 18, 2010 at 12:47 PM, Frans Meulenbroeks <
fransmeulenbroeks@gmail.com> wrote:

> 2010/7/16 Richard Purdie <rpurdie@rpsys.net>
>
> > Whilst our layers mechanism, is great it does have a drawback which has
> > bugged me for a while. If you have a recipe like pointercal which has
> > machine specific information in it and you have your new machine code in
> > a layer, how do you add a pointercal file for your machine?
> >
> > Answer is you copy the whole pointercal recipe and files into your
> > layer, then add the single file for your machine. To me this is ugly,
> > ugly, ugly. We hate code duplication and as soon as you create two
> > copies of the same information, we've failed.
> >
> > So how could we do this better? Somehow we need to say that a given
> > directory X has some information which should be merged with the
> > original recipe. I've thought through several different ways of doing
> > this and the best solution I found was "bbappend".
> >
> > The idea is that if bitbake finds any X.bbappend files, when it loads
> > X.bb, it will also include these files after it parses the base .bb file
> > (but before finalise and the anonymous methods run). This means that
> > the .bbappend file can poke around and do whatever it might want to the
> > recipe to customise it.
> >
> > I went ahead and tried it out as its quite simple to code this in
> > bitbake. I liked the result enough I've already merged this into Poky:
> >
> >
> >
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> >
> > I'm proposing to push it to bitbake master if there are no serious
> > objections.
> >
> > As an example use case, for the pointercal case above in another later
> > you could add a pointercal_0.0.bbappend file which contained something
> > like:
> >
> > FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d,
> > True))}"
> >
> > which would then cause the directory containing the bbappend file to be
> > searched for pointercal files.
> >
> > There are of course many other uses this could be put to for creating
> > customised layers, its totally generic.
> >
> > For the specific case of paths, I have wondered if there would be a way
> > to leverage help from bitbake in creating a sane set of search paths but
> > I'm still thinking about that. This extension is good enough in its own
> > right in my opinion to be worthwhile.
> >
> > Cheers,
> >
> > Richard
> >
>
> I've been pondering a litlte bit more about this.
> Will the .bbappend mechanism allow me to override a file in the original
> recipe?
> If so how?
>
> Typical use case: I want to have a dedicated defconfig for my linux kernel.
> Alternate use case: a dedicated busybox configuration.
>

You can do this today with FILESPATHBASE.

I.e. FILESPATHBASE =. "${TOPDIR}/files:", then mkdir -p files/busybox; cp
mybusyboxdefconfig files/busybox/defconfig
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Frans Meulenbroeks]

2010/7/18 Chris Larson <clarson@kergoth.com>

> On Sun, Jul 18, 2010 at 12:47 PM, Frans Meulenbroeks <
> fransmeulenbroeks@gmail.com> wrote:
>
> > 2010/7/16 Richard Purdie <rpurdie@rpsys.net>
> >
> > > Whilst our layers mechanism, is great it does have a drawback which has
> > > bugged me for a while. If you have a recipe like pointercal which has
> > > machine specific information in it and you have your new machine code
> in
> > > a layer, how do you add a pointercal file for your machine?
> > >
> > > Answer is you copy the whole pointercal recipe and files into your
> > > layer, then add the single file for your machine. To me this is ugly,
> > > ugly, ugly. We hate code duplication and as soon as you create two
> > > copies of the same information, we've failed.
> > >
> > > So how could we do this better? Somehow we need to say that a given
> > > directory X has some information which should be merged with the
> > > original recipe. I've thought through several different ways of doing
> > > this and the best solution I found was "bbappend".
> > >
> > > The idea is that if bitbake finds any X.bbappend files, when it loads
> > > X.bb, it will also include these files after it parses the base .bb
> file
> > > (but before finalise and the anonymous methods run). This means that
> > > the .bbappend file can poke around and do whatever it might want to the
> > > recipe to customise it.
> > >
> > > I went ahead and tried it out as its quite simple to code this in
> > > bitbake. I liked the result enough I've already merged this into Poky:
> > >
> > >
> > >
> >
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> > >
> > > I'm proposing to push it to bitbake master if there are no serious
> > > objections.
> > >
> > > As an example use case, for the pointercal case above in another later
> > > you could add a pointercal_0.0.bbappend file which contained something
> > > like:
> > >
> > > FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d,
> > > True))}"
> > >
> > > which would then cause the directory containing the bbappend file to be
> > > searched for pointercal files.
> > >
> > > There are of course many other uses this could be put to for creating
> > > customised layers, its totally generic.
> > >
> > > For the specific case of paths, I have wondered if there would be a way
> > > to leverage help from bitbake in creating a sane set of search paths
> but
> > > I'm still thinking about that. This extension is good enough in its own
> > > right in my opinion to be worthwhile.
> > >
> > > Cheers,
> > >
> > > Richard
> > >
> >
> > I've been pondering a litlte bit more about this.
> > Will the .bbappend mechanism allow me to override a file in the original
> > recipe?
> > If so how?
> >
> > Typical use case: I want to have a dedicated defconfig for my linux
> kernel.
> > Alternate use case: a dedicated busybox configuration.
> >
>
> You can do this today with FILESPATHBASE.
>
> I.e. FILESPATHBASE =. "${TOPDIR}/files:", then mkdir -p files/busybox; cp
> mybusyboxdefconfig files/busybox/defconfig
>

Ah cool, didn't know that, will give it a try.
Actually this might be mentioned somewhere in the wiki (if it is not in
yet). I can imagine especially wrt kernel that people want to do have their
own config.

Frans

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Detlef Vollmann]

On 07/18/10 22:25, Richard Purdie wrote:
> On Sun, 2010-07-18 at 21:24 +0200, Detlef Vollmann wrote:
>> On 07/18/10 18:06, Chris Larson wrote:
>>> On Sun, Jul 18, 2010 at 9:01 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
>>>
>>>> On 07/18/10 17:13, Chris Larson wrote:
>>>>
>>>>> On Sun, Jul 18, 2010 at 3:06 AM, Detlef Vollmann <dv@vollmann.ch> wrote:
>>>>>
>>>>>  On 07/16/10 16:25, Richard Purdie wrote:
>>>>>>  For the specific case of paths, I have wondered if there would be a way
>>>>>>> to leverage help from bitbake in creating a sane set of search paths but
>>>>>>> I'm still thinking about that. This extension is good enough in its own
>>>>>>> right in my opinion to be worthwhile.
>>>>>>>
>>>>>>>  This might be true, but solving the general path problem would be
>>>>>> much more useful.
>>>>>> If you want to add a new version of e.g. gcc in your ovlerlay,
>>>>>> you have to pull over the whole gcc directory with all its .inc classes.
>>>>>> This can't be solved by .bbapend, or am I missing something?
>>>>>>
>>>>> You can handle that today by adding to FILESPATHBASE.  All file:// files
>>>>> from SRC_URI are located using FILESPATH, which is constructed from
>>>>> FILESPATHBASE, FILESPATHPKG, and OVERRIDES.
>>>>>
>>>> AFAIK this doen't work for include/require :-(
>>>
>>> Don't copy the recipe at all, use .bbappend instead, and you won't need to
>>> have the .inc in a different collection from the recipe itself.
>> In that case I didn't understand .bbappend.
>> If I have a recipe gcc_4.3.2.bb in e.g. oe-stable/recipes/gcc, but in
>> that directory is no gcc_4.4.4.bb,  and I put gcc_4.4.4.bbappend into
>> myproject/recipes/gcc, that would probably given an error like
>> 'no .bb file to append'.
>> At least that's how I understood .bbappend.
>>
>>> Alternatively, you can adjust BBPATH to also include the other collection
>>> based paths + os.path.dirname('${FILE_DIRNAME}') or so.
>> I'd be very interested to find a way so that a 'require gcc.inc' in
>> myproject/recipes/gcc_4.4.4.bb would find oe-stable/recipes/gcc/gcc.inc.
>> So far I failed.
> 
> Does something like "require recipes/gcc/gcc.inc" not just work already
> as it will search BBPATH...
Thanks, that actually works. Great :-)

Thanks again,
   Detlef

Re: A new bitbake extension: .bbappend files

[Douglas Royds]

On 17/07/10 02:25, Richard Purdie wrote:
> Whilst our layers mechanism, is great it does have a drawback which has
> bugged me for a while. If you have a recipe like pointercal which has
> machine specific information in it and you have your new machine code in
> a layer, how do you add a pointercal file for your machine?
>
> Answer is you copy the whole pointercal recipe and files into your
> layer, then add the single file for your machine.
>    

For our overlay distro, I solved this problem as follows (in 
"conf/distro/tait.conf"):

# Compute a FILESPATHBASE that will cause the FILESPATH search string 
(for file:// URIs) to
# list tait.openembedded directories first, and org.openembedded 
directories second.
# This allows us to put a single file, eg. lighttpd.conf, into our 
tait.openembedded distribution,
# and it will be picked up by OE in preference to the deault lighttpd.conf.
# The directory containing the recipe is not necessarily the same as the 
package name.
# For instance, for lighttpd_1.2.3.bb, we want
# FILE_DIRBASENAME = lighttpd
# FILESPATHBASE = 
"/home/roydsd/Work/DBS/tait.openembedded/branches/2.6.32/recipes/lighttpd:
# 
/home/roydsd/Work/DBS/org.openembedded.dev/branches/2010-02-23/recipes/lighttpd"
# BBFILES is a space-separated list of /path/to/recipes/*/*.bb patterns.
# Split each pattern at the recipes keyword to get 
/home/roydsd/Work/DBS/tait.oe/trunk/,
# then tack on recipes/lighttpd, and
# join the whole lot together with colons
FILE_DIRBASENAME = 
"${@os.path.basename(os.path.dirname(bb.data.getVar('FILE', d)))}"
FILESPATHBASE = ${@':'.join([ pattern.split('recipes')[0] + 
'recipes/${FILE_DIRBASENAME}' for pattern in '${BBFILES}'.split() ])}

For example:

tait.openembedded/trunk/recipes/ntp/
└── files/
└── ntp.conf

BBFILES="/home/roydsd/Work/DBS/tait.openembedded/trunk/recipes/*/*.bb 
/home/roydsd/Work/DBS/org.openembedded.dev/branches/2010-02-23/recipes/*/*.bb"
FILESPATHBASE="/home/roydsd/Work/DBS/tait.openembedded/trunk/recipes/ntp:/home/roydsd/Work/DBS/org.openembedded.dev/branches/2010-02-23/recipes/ntp"

Does this achieve what you had in mind?
Douglas.
=======================================================================
This email, including any attachments, is only for the intended
addressee.  It is subject to copyright, is confidential and may be
the subject of legal or other privilege, none of which is waived or
lost by reason of this transmission.
If the receiver is not the intended addressee, please accept our
apologies, notify us by return, delete all copies and perform no
other act on the email.
Unfortunately, we cannot warrant that the email has not been
altered or corrupted during transmission.
=======================================================================

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Hauser, Wolfgang (external)]

>I'd be very interested to find a way so that a 'require gcc.inc' in
>myproject/recipes/gcc_4.4.4.bb would find
oe-stable/recipes/gcc/gcc.inc.
>So far I failed.

Try to add the relative path of the .inc to your require statement:
'require recipes/gcc/gcc.inc'
(see also the arago overlays as example) my overlays are working in this
way.
The problem was, all require statements should be done in this way, so
the building of overlays would be much easier.

Wolfgang

Re: A new bitbake extension: .bbappend files

[Tom Rini]

Richard Purdie wrote:
> Whilst our layers mechanism, is great it does have a drawback which has
> bugged me for a while. If you have a recipe like pointercal which has
> machine specific information in it and you have your new machine code in
> a layer, how do you add a pointercal file for your machine?
> 
> Answer is you copy the whole pointercal recipe and files into your
> layer, then add the single file for your machine. To me this is ugly,
> ugly, ugly. We hate code duplication and as soon as you create two
> copies of the same information, we've failed.
> 
> So how could we do this better? Somehow we need to say that a given
> directory X has some information which should be merged with the
> original recipe. I've thought through several different ways of doing
> this and the best solution I found was "bbappend".
> 
> The idea is that if bitbake finds any X.bbappend files, when it loads
> X.bb, it will also include these files after it parses the base .bb file
> (but before finalise and the anonymous methods run). This means that
> the .bbappend file can poke around and do whatever it might want to the
> recipe to customise it.
> 
> I went ahead and tried it out as its quite simple to code this in
> bitbake. I liked the result enough I've already merged this into Poky:
> 
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> 
> I'm proposing to push it to bitbake master if there are no serious
> objections.
> 
> As an example use case, for the pointercal case above in another later
> you could add a pointercal_0.0.bbappend file which contained something
> like:
> 
> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d, True))}"
> 
> which would then cause the directory containing the bbappend file to be
> searched for pointercal files.
> 
> There are of course many other uses this could be put to for creating
> customised layers, its totally generic.
> 
> For the specific case of paths, I have wondered if there would be a way
> to leverage help from bitbake in creating a sane set of search paths but
> I'm still thinking about that. This extension is good enough in its own
> right in my opinion to be worthwhile.

I must be missing something.  How is this better than amend.inc where 
today you would just do:
# Just to make sure PR does change, could actually be omitted for this
# example
$ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
$ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called

?  Or did you just give too trivial of an example here, and as there are 
indeed places where amend.inc falls down today, this does work?

-- 
Tom Rini
Mentor Graphics Corporation

Re: A new bitbake extension: .bbappend files

[Chris Larson]

On Mon, Jul 19, 2010 at 1:50 PM, Tom Rini <tom_rini@mentor.com> wrote:

> Richard Purdie wrote:
>
>> Whilst our layers mechanism, is great it does have a drawback which has
>> bugged me for a while. If you have a recipe like pointercal which has
>> machine specific information in it and you have your new machine code in
>> a layer, how do you add a pointercal file for your machine?
>>
>> Answer is you copy the whole pointercal recipe and files into your
>> layer, then add the single file for your machine. To me this is ugly,
>> ugly, ugly. We hate code duplication and as soon as you create two
>> copies of the same information, we've failed.
>>
>> So how could we do this better? Somehow we need to say that a given
>> directory X has some information which should be merged with the
>> original recipe. I've thought through several different ways of doing
>> this and the best solution I found was "bbappend".
>>
>> The idea is that if bitbake finds any X.bbappend files, when it loads
>> X.bb, it will also include these files after it parses the base .bb file
>> (but before finalise and the anonymous methods run). This means that
>> the .bbappend file can poke around and do whatever it might want to the
>> recipe to customise it.
>>
>> I went ahead and tried it out as its quite simple to code this in
>> bitbake. I liked the result enough I've already merged this into Poky:
>>
>>
>> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
>>
>> I'm proposing to push it to bitbake master if there are no serious
>> objections.
>>
>> As an example use case, for the pointercal case above in another later
>> you could add a pointercal_0.0.bbappend file which contained something
>> like:
>>
>> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d,
>> True))}"
>>
>> which would then cause the directory containing the bbappend file to be
>> searched for pointercal files.
>>
>> There are of course many other uses this could be put to for creating
>> customised layers, its totally generic.
>>
>> For the specific case of paths, I have wondered if there would be a way
>> to leverage help from bitbake in creating a sane set of search paths but
>> I'm still thinking about that. This extension is good enough in its own
>> right in my opinion to be worthwhile.
>>
>
> I must be missing something.  How is this better than amend.inc where today
> you would just do:
> # Just to make sure PR does change, could actually be omitted for this
> # example
> $ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
> $ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called
>
> ?  Or did you just give too trivial of an example here, and as there are
> indeed places where amend.inc falls down today, this does work?


It's the opposite.  amend.inc is a bit more flexible than bbappend, not vice
versa, but bbappend has fewer performance implications, and is supported
directly by bitbake itself (and ensures that anonymous python functions will
work fine in appended metadata).  Aside: I also suspect that BBCLASSEXTEND
alterations will work correctly from a bbappend, which does not work
correctly today from an amend.inc.

You'd have to be careful to lock down your appended versions when using
bbappend, since bitbake could easily pick a newer version without the append
out from under you, which is not an issue with amend.inc. This is why I was
planning to test it out with some real world situations to see if it's
sufficiently flexible for most cases or not.  I haven't yet had a chance.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Chris Larson]

On Mon, Jul 19, 2010 at 2:22 PM, Chris Larson <clarson@kergoth.com> wrote:

>
>
> On Mon, Jul 19, 2010 at 1:50 PM, Tom Rini <tom_rini@mentor.com> wrote:
>
>> Richard Purdie wrote:
>>
>>> Whilst our layers mechanism, is great it does have a drawback which has
>>> bugged me for a while. If you have a recipe like pointercal which has
>>> machine specific information in it and you have your new machine code in
>>> a layer, how do you add a pointercal file for your machine?
>>>
>>> Answer is you copy the whole pointercal recipe and files into your
>>> layer, then add the single file for your machine. To me this is ugly,
>>> ugly, ugly. We hate code duplication and as soon as you create two
>>> copies of the same information, we've failed.
>>>
>>> So how could we do this better? Somehow we need to say that a given
>>> directory X has some information which should be merged with the
>>> original recipe. I've thought through several different ways of doing
>>> this and the best solution I found was "bbappend".
>>>
>>> The idea is that if bitbake finds any X.bbappend files, when it loads
>>> X.bb, it will also include these files after it parses the base .bb file
>>> (but before finalise and the anonymous methods run). This means that
>>> the .bbappend file can poke around and do whatever it might want to the
>>> recipe to customise it.
>>>
>>> I went ahead and tried it out as its quite simple to code this in
>>> bitbake. I liked the result enough I've already merged this into Poky:
>>>
>>>
>>> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
>>>
>>> I'm proposing to push it to bitbake master if there are no serious
>>> objections.
>>>
>>> As an example use case, for the pointercal case above in another later
>>> you could add a pointercal_0.0.bbappend file which contained something
>>> like:
>>>
>>> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d,
>>> True))}"
>>>
>>> which would then cause the directory containing the bbappend file to be
>>> searched for pointercal files.
>>>
>>> There are of course many other uses this could be put to for creating
>>> customised layers, its totally generic.
>>>
>>> For the specific case of paths, I have wondered if there would be a way
>>> to leverage help from bitbake in creating a sane set of search paths but
>>> I'm still thinking about that. This extension is good enough in its own
>>> right in my opinion to be worthwhile.
>>>
>>
>> I must be missing something.  How is this better than amend.inc where
>> today you would just do:
>> # Just to make sure PR does change, could actually be omitted for this
>> # example
>> $ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
>> $ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called
>>
>> ?  Or did you just give too trivial of an example here, and as there are
>> indeed places where amend.inc falls down today, this does work?
>
>
> It's the opposite.  amend.inc is a bit more flexible than bbappend, not
> vice versa, but bbappend has fewer performance implications, and is
> supported directly by bitbake itself (and ensures that anonymous python
> functions will work fine in appended metadata).  Aside: I also suspect that
> BBCLASSEXTEND alterations will work correctly from a bbappend, which does
> not work correctly today from an amend.inc.
>
> You'd have to be careful to lock down your appended versions when using
> bbappend, since bitbake could easily pick a newer version without the append
> out from under you, which is not an issue with amend.inc.
>

Although, I suppose you could set DEFAULT_PREFERENCE in the bbappend, so
whenever you include that collection, it gets preferred.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Frans Meulenbroeks]

2010/7/19 Chris Larson <clarson@kergoth.com>

> On Mon, Jul 19, 2010 at 2:22 PM, Chris Larson <clarson@kergoth.com> wrote:
>
> >
> >
> > On Mon, Jul 19, 2010 at 1:50 PM, Tom Rini <tom_rini@mentor.com> wrote:
> >
> >> Richard Purdie wrote:
> >>
> >>> Whilst our layers mechanism, is great it does have a drawback which has
> >>> bugged me for a while. If you have a recipe like pointercal which has
> >>> machine specific information in it and you have your new machine code
> in
> >>> a layer, how do you add a pointercal file for your machine?
> >>>
> >>> Answer is you copy the whole pointercal recipe and files into your
> >>> layer, then add the single file for your machine. To me this is ugly,
> >>> ugly, ugly. We hate code duplication and as soon as you create two
> >>> copies of the same information, we've failed.
> >>>
> >>> So how could we do this better? Somehow we need to say that a given
> >>> directory X has some information which should be merged with the
> >>> original recipe. I've thought through several different ways of doing
> >>> this and the best solution I found was "bbappend".
> >>>
> >>> The idea is that if bitbake finds any X.bbappend files, when it loads
> >>> X.bb, it will also include these files after it parses the base .bb
> file
> >>> (but before finalise and the anonymous methods run). This means that
> >>> the .bbappend file can poke around and do whatever it might want to the
> >>> recipe to customise it.
> >>>
> >>> I went ahead and tried it out as its quite simple to code this in
> >>> bitbake. I liked the result enough I've already merged this into Poky:
> >>>
> >>>
> >>>
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> >>>
> >>> I'm proposing to push it to bitbake master if there are no serious
> >>> objections.
> >>>
> >>> As an example use case, for the pointercal case above in another later
> >>> you could add a pointercal_0.0.bbappend file which contained something
> >>> like:
> >>>
> >>> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE', d,
> >>> True))}"
> >>>
> >>> which would then cause the directory containing the bbappend file to be
> >>> searched for pointercal files.
> >>>
> >>> There are of course many other uses this could be put to for creating
> >>> customised layers, its totally generic.
> >>>
> >>> For the specific case of paths, I have wondered if there would be a way
> >>> to leverage help from bitbake in creating a sane set of search paths
> but
> >>> I'm still thinking about that. This extension is good enough in its own
> >>> right in my opinion to be worthwhile.
> >>>
> >>
> >> I must be missing something.  How is this better than amend.inc where
> >> today you would just do:
> >> # Just to make sure PR does change, could actually be omitted for this
> >> # example
> >> $ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
> >> $ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called
> >>
> >> ?  Or did you just give too trivial of an example here, and as there are
> >> indeed places where amend.inc falls down today, this does work?
> >
> >
> > It's the opposite.  amend.inc is a bit more flexible than bbappend, not
> > vice versa, but bbappend has fewer performance implications, and is
> > supported directly by bitbake itself (and ensures that anonymous python
> > functions will work fine in appended metadata).  Aside: I also suspect
> that
> > BBCLASSEXTEND alterations will work correctly from a bbappend, which does
> > not work correctly today from an amend.inc.
> >
> > You'd have to be careful to lock down your appended versions when using
> > bbappend, since bitbake could easily pick a newer version without the
> append
> > out from under you, which is not an issue with amend.inc.
> >
>
> Although, I suppose you could set DEFAULT_PREFERENCE in the bbappend, so
> whenever you include that collection, it gets preferred.
>
> Not if your distro decides to move to a new version. The distro choice gets
priority to DEFAULT_PREFERENCE.

Btw is there any doc on how to use amend.inc? Looks like a neat thing.

Frans

Re: A new bitbake extension: .bbappend files

[Chris Larson]

On Tue, Jul 20, 2010 at 12:28 AM, Frans Meulenbroeks <
fransmeulenbroeks@gmail.com> wrote:

> 2010/7/19 Chris Larson <clarson@kergoth.com>
>
> > On Mon, Jul 19, 2010 at 2:22 PM, Chris Larson <clarson@kergoth.com>
> wrote:
> >
> > >
> > >
> > > On Mon, Jul 19, 2010 at 1:50 PM, Tom Rini <tom_rini@mentor.com> wrote:
> > >
> > >> Richard Purdie wrote:
> > >>
> > >>> Whilst our layers mechanism, is great it does have a drawback which
> has
> > >>> bugged me for a while. If you have a recipe like pointercal which has
> > >>> machine specific information in it and you have your new machine code
> > in
> > >>> a layer, how do you add a pointercal file for your machine?
> > >>>
> > >>> Answer is you copy the whole pointercal recipe and files into your
> > >>> layer, then add the single file for your machine. To me this is ugly,
> > >>> ugly, ugly. We hate code duplication and as soon as you create two
> > >>> copies of the same information, we've failed.
> > >>>
> > >>> So how could we do this better? Somehow we need to say that a given
> > >>> directory X has some information which should be merged with the
> > >>> original recipe. I've thought through several different ways of doing
> > >>> this and the best solution I found was "bbappend".
> > >>>
> > >>> The idea is that if bitbake finds any X.bbappend files, when it loads
> > >>> X.bb, it will also include these files after it parses the base .bb
> > file
> > >>> (but before finalise and the anonymous methods run). This means that
> > >>> the .bbappend file can poke around and do whatever it might want to
> the
> > >>> recipe to customise it.
> > >>>
> > >>> I went ahead and tried it out as its quite simple to code this in
> > >>> bitbake. I liked the result enough I've already merged this into
> Poky:
> > >>>
> > >>>
> > >>>
> >
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> > >>>
> > >>> I'm proposing to push it to bitbake master if there are no serious
> > >>> objections.
> > >>>
> > >>> As an example use case, for the pointercal case above in another
> later
> > >>> you could add a pointercal_0.0.bbappend file which contained
> something
> > >>> like:
> > >>>
> > >>> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE',
> d,
> > >>> True))}"
> > >>>
> > >>> which would then cause the directory containing the bbappend file to
> be
> > >>> searched for pointercal files.
> > >>>
> > >>> There are of course many other uses this could be put to for creating
> > >>> customised layers, its totally generic.
> > >>>
> > >>> For the specific case of paths, I have wondered if there would be a
> way
> > >>> to leverage help from bitbake in creating a sane set of search paths
> > but
> > >>> I'm still thinking about that. This extension is good enough in its
> own
> > >>> right in my opinion to be worthwhile.
> > >>>
> > >>
> > >> I must be missing something.  How is this better than amend.inc where
> > >> today you would just do:
> > >> # Just to make sure PR does change, could actually be omitted for this
> > >> # example
> > >> $ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
> > >> $ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called
> > >>
> > >> ?  Or did you just give too trivial of an example here, and as there
> are
> > >> indeed places where amend.inc falls down today, this does work?
> > >
> > >
> > > It's the opposite.  amend.inc is a bit more flexible than bbappend, not
> > > vice versa, but bbappend has fewer performance implications, and is
> > > supported directly by bitbake itself (and ensures that anonymous python
> > > functions will work fine in appended metadata).  Aside: I also suspect
> > that
> > > BBCLASSEXTEND alterations will work correctly from a bbappend, which
> does
> > > not work correctly today from an amend.inc.
> > >
> > > You'd have to be careful to lock down your appended versions when using
> > > bbappend, since bitbake could easily pick a newer version without the
> > append
> > > out from under you, which is not an issue with amend.inc.
> > >
> >
> > Although, I suppose you could set DEFAULT_PREFERENCE in the bbappend, so
> > whenever you include that collection, it gets preferred.
> >
> > Not if your distro decides to move to a new version. The distro choice
> gets
> priority to DEFAULT_PREFERENCE.
>

Fair point, best to use a version lockdown tool if you leverage bbappend, at
this time, so there's a bit more work involved.

Btw is there any doc on how to use amend.inc? Looks like a neat thing.


I don't think so, but there isn't much to document.  Just add INHERIT +=
"amend", and then you can place a "amend.inc" file anywhere in any recipe's
FILESPATH, and it will be parsed and added onto the metadata of that recipe.
 Avoid modifying BBCLASSEXTEND from it, that's known to have issues.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Tom Rini]

Chris Larson wrote:
> On Tue, Jul 20, 2010 at 12:28 AM, Frans Meulenbroeks <
> fransmeulenbroeks@gmail.com> wrote:
> 
>> 2010/7/19 Chris Larson <clarson@kergoth.com>
>>
>>> On Mon, Jul 19, 2010 at 2:22 PM, Chris Larson <clarson@kergoth.com>
>> wrote:
>>>>
>>>> On Mon, Jul 19, 2010 at 1:50 PM, Tom Rini <tom_rini@mentor.com> wrote:
>>>>
>>>>> Richard Purdie wrote:
>>>>>
>>>>>> Whilst our layers mechanism, is great it does have a drawback which
>> has
>>>>>> bugged me for a while. If you have a recipe like pointercal which has
>>>>>> machine specific information in it and you have your new machine code
>>> in
>>>>>> a layer, how do you add a pointercal file for your machine?
>>>>>>
>>>>>> Answer is you copy the whole pointercal recipe and files into your
>>>>>> layer, then add the single file for your machine. To me this is ugly,
>>>>>> ugly, ugly. We hate code duplication and as soon as you create two
>>>>>> copies of the same information, we've failed.
>>>>>>
>>>>>> So how could we do this better? Somehow we need to say that a given
>>>>>> directory X has some information which should be merged with the
>>>>>> original recipe. I've thought through several different ways of doing
>>>>>> this and the best solution I found was "bbappend".
>>>>>>
>>>>>> The idea is that if bitbake finds any X.bbappend files, when it loads
>>>>>> X.bb, it will also include these files after it parses the base .bb
>>> file
>>>>>> (but before finalise and the anonymous methods run). This means that
>>>>>> the .bbappend file can poke around and do whatever it might want to
>> the
>>>>>> recipe to customise it.
>>>>>>
>>>>>> I went ahead and tried it out as its quite simple to code this in
>>>>>> bitbake. I liked the result enough I've already merged this into
>> Poky:
>>>>>>
>>>>>>
>> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
>>>>>> I'm proposing to push it to bitbake master if there are no serious
>>>>>> objections.
>>>>>>
>>>>>> As an example use case, for the pointercal case above in another
>> later
>>>>>> you could add a pointercal_0.0.bbappend file which contained
>> something
>>>>>> like:
>>>>>>
>>>>>> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE',
>> d,
>>>>>> True))}"
>>>>>>
>>>>>> which would then cause the directory containing the bbappend file to
>> be
>>>>>> searched for pointercal files.
>>>>>>
>>>>>> There are of course many other uses this could be put to for creating
>>>>>> customised layers, its totally generic.
>>>>>>
>>>>>> For the specific case of paths, I have wondered if there would be a
>> way
>>>>>> to leverage help from bitbake in creating a sane set of search paths
>>> but
>>>>>> I'm still thinking about that. This extension is good enough in its
>> own
>>>>>> right in my opinion to be worthwhile.
>>>>>>
>>>>> I must be missing something.  How is this better than amend.inc where
>>>>> today you would just do:
>>>>> # Just to make sure PR does change, could actually be omitted for this
>>>>> # example
>>>>> $ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
>>>>> $ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called
>>>>>
>>>>> ?  Or did you just give too trivial of an example here, and as there
>> are
>>>>> indeed places where amend.inc falls down today, this does work?
>>>>
>>>> It's the opposite.  amend.inc is a bit more flexible than bbappend, not
>>>> vice versa, but bbappend has fewer performance implications, and is
>>>> supported directly by bitbake itself (and ensures that anonymous python
>>>> functions will work fine in appended metadata).  Aside: I also suspect
>>> that
>>>> BBCLASSEXTEND alterations will work correctly from a bbappend, which
>> does
>>>> not work correctly today from an amend.inc.
>>>>
>>>> You'd have to be careful to lock down your appended versions when using
>>>> bbappend, since bitbake could easily pick a newer version without the
>>> append
>>>> out from under you, which is not an issue with amend.inc.
>>>>
>>> Although, I suppose you could set DEFAULT_PREFERENCE in the bbappend, so
>>> whenever you include that collection, it gets preferred.
>>>
>>> Not if your distro decides to move to a new version. The distro choice
>> gets
>> priority to DEFAULT_PREFERENCE.
>>
> 
> Fair point, best to use a version lockdown tool if you leverage bbappend, at
> this time, so there's a bit more work involved.
> 
> Btw is there any doc on how to use amend.inc? Looks like a neat thing.
> 
> 
> I don't think so, but there isn't much to document.  Just add INHERIT +=
> "amend", and then you can place a "amend.inc" file anywhere in any recipe's
> FILESPATH, and it will be parsed and added onto the metadata of that recipe.
>  Avoid modifying BBCLASSEXTEND from it, that's known to have issues.

It's also needed, in some cases, to lock down your changes via OVERRIDES 
that will only match what you want to work on.  For example, we're using 
a snapshot with an older python, so amend.inc in our python directory 
looks like:

SRC_URI_append_pn-python-native = 
"file://python-2.6-r73342.patch;patch=1;pnum=0"
PARALLEL_MAKE_pn-python-native = ""
PR_append_pn-python-native .= ".1"

So that we only change python-native and not all of the other python 
recipes.  This would indeed not be an issue with bbappend as we'd have 
put all of that into python-native_2.6.1.bbappend (but it sounds like we 
would have had to add something for FILESPATH manipulation?  If so, that 
sounds really bad to me, Richard?).

-- 
Tom Rini
Mentor Graphics Corporation

Re: A new bitbake extension: .bbappend files

[Tom Rini]

Tom Rini wrote:
> Chris Larson wrote:
>> On Tue, Jul 20, 2010 at 12:28 AM, Frans Meulenbroeks <
>> fransmeulenbroeks@gmail.com> wrote:
>>
>>> 2010/7/19 Chris Larson <clarson@kergoth.com>
>>>
>>>> On Mon, Jul 19, 2010 at 2:22 PM, Chris Larson <clarson@kergoth.com>
>>> wrote:
>>>>>
>>>>> On Mon, Jul 19, 2010 at 1:50 PM, Tom Rini <tom_rini@mentor.com> wrote:
>>>>>
>>>>>> Richard Purdie wrote:
>>>>>>
>>>>>>> Whilst our layers mechanism, is great it does have a drawback which
>>> has
>>>>>>> bugged me for a while. If you have a recipe like pointercal which 
>>>>>>> has
>>>>>>> machine specific information in it and you have your new machine 
>>>>>>> code
>>>> in
>>>>>>> a layer, how do you add a pointercal file for your machine?
>>>>>>>
>>>>>>> Answer is you copy the whole pointercal recipe and files into your
>>>>>>> layer, then add the single file for your machine. To me this is 
>>>>>>> ugly,
>>>>>>> ugly, ugly. We hate code duplication and as soon as you create two
>>>>>>> copies of the same information, we've failed.
>>>>>>>
>>>>>>> So how could we do this better? Somehow we need to say that a given
>>>>>>> directory X has some information which should be merged with the
>>>>>>> original recipe. I've thought through several different ways of 
>>>>>>> doing
>>>>>>> this and the best solution I found was "bbappend".
>>>>>>>
>>>>>>> The idea is that if bitbake finds any X.bbappend files, when it 
>>>>>>> loads
>>>>>>> X.bb, it will also include these files after it parses the base .bb
>>>> file
>>>>>>> (but before finalise and the anonymous methods run). This means that
>>>>>>> the .bbappend file can poke around and do whatever it might want to
>>> the
>>>>>>> recipe to customise it.
>>>>>>>
>>>>>>> I went ahead and tried it out as its quite simple to code this in
>>>>>>> bitbake. I liked the result enough I've already merged this into
>>> Poky:
>>>>>>>
>>>>>>>
>>> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660 
>>>
>>>>>>> I'm proposing to push it to bitbake master if there are no serious
>>>>>>> objections.
>>>>>>>
>>>>>>> As an example use case, for the pointercal case above in another
>>> later
>>>>>>> you could add a pointercal_0.0.bbappend file which contained
>>> something
>>>>>>> like:
>>>>>>>
>>>>>>> FILESPATH := "${FILESPATH}:${@os.path.dirname(bb.data.getVar('FILE',
>>> d,
>>>>>>> True))}"
>>>>>>>
>>>>>>> which would then cause the directory containing the bbappend file to
>>> be
>>>>>>> searched for pointercal files.
>>>>>>>
>>>>>>> There are of course many other uses this could be put to for 
>>>>>>> creating
>>>>>>> customised layers, its totally generic.
>>>>>>>
>>>>>>> For the specific case of paths, I have wondered if there would be a
>>> way
>>>>>>> to leverage help from bitbake in creating a sane set of search paths
>>>> but
>>>>>>> I'm still thinking about that. This extension is good enough in its
>>> own
>>>>>>> right in my opinion to be worthwhile.
>>>>>>>
>>>>>> I must be missing something.  How is this better than amend.inc where
>>>>>> today you would just do:
>>>>>> # Just to make sure PR does change, could actually be omitted for 
>>>>>> this
>>>>>> # example
>>>>>> $ echo 'PR .= ".1"' > mymachine/recipes/pointercal/amend.inc
>>>>>> $ cp /tmp/mycal mymachine/recipes/pointercal/whatever_it_is_called
>>>>>>
>>>>>> ?  Or did you just give too trivial of an example here, and as there
>>> are
>>>>>> indeed places where amend.inc falls down today, this does work?
>>>>>
>>>>> It's the opposite.  amend.inc is a bit more flexible than bbappend, 
>>>>> not
>>>>> vice versa, but bbappend has fewer performance implications, and is
>>>>> supported directly by bitbake itself (and ensures that anonymous 
>>>>> python
>>>>> functions will work fine in appended metadata).  Aside: I also suspect
>>>> that
>>>>> BBCLASSEXTEND alterations will work correctly from a bbappend, which
>>> does
>>>>> not work correctly today from an amend.inc.
>>>>>
>>>>> You'd have to be careful to lock down your appended versions when 
>>>>> using
>>>>> bbappend, since bitbake could easily pick a newer version without the
>>>> append
>>>>> out from under you, which is not an issue with amend.inc.
>>>>>
>>>> Although, I suppose you could set DEFAULT_PREFERENCE in the 
>>>> bbappend, so
>>>> whenever you include that collection, it gets preferred.
>>>>
>>>> Not if your distro decides to move to a new version. The distro choice
>>> gets
>>> priority to DEFAULT_PREFERENCE.
>>>
>>
>> Fair point, best to use a version lockdown tool if you leverage 
>> bbappend, at
>> this time, so there's a bit more work involved.
>>
>> Btw is there any doc on how to use amend.inc? Looks like a neat thing.
>>
>>
>> I don't think so, but there isn't much to document.  Just add INHERIT +=
>> "amend", and then you can place a "amend.inc" file anywhere in any 
>> recipe's
>> FILESPATH, and it will be parsed and added onto the metadata of that 
>> recipe.
>>  Avoid modifying BBCLASSEXTEND from it, that's known to have issues.
> 
> It's also needed, in some cases, to lock down your changes via OVERRIDES 
> that will only match what you want to work on.  For example, we're using 
> a snapshot with an older python, so amend.inc in our python directory 
> looks like:
> 
> SRC_URI_append_pn-python-native = 
> "file://python-2.6-r73342.patch;patch=1;pnum=0"
> PARALLEL_MAKE_pn-python-native = ""
> PR_append_pn-python-native .= ".1"
> 
> So that we only change python-native and not all of the other python 
> recipes.  This would indeed not be an issue with bbappend as we'd have 
> put all of that into python-native_2.6.1.bbappend (but it sounds like we 
> would have had to add something for FILESPATH manipulation?  If so, that 
> sounds really bad to me, Richard?).

OK, misrecollection on my part.  amend.inc needs you to do some form of 
mucking as well (if you want to add files or patches or ..) and in both 
cases you could stick it into local.conf.

-- 
Tom Rini
Mentor Graphics Corporation

Re: A new bitbake extension: .bbappend files

[Michael Smith]

Tom Rini wrote:
> It's also needed, in some cases, to lock down your changes via OVERRIDES 
> that will only match what you want to work on.  For example, we're using 
> a snapshot with an older python, so amend.inc in our python directory 
> looks like:
> 
> SRC_URI_append_pn-python-native = 
> "file://python-2.6-r73342.patch;patch=1;pnum=0"
> PARALLEL_MAKE_pn-python-native = ""
> PR_append_pn-python-native .= ".1"
> 
> So that we only change python-native and not all of the other python 
> recipes.

I think you could get the same effect by putting your amend.inc in a 
python-native subdirectory under your overlay recipes/python directory.

Maybe there's some native magic going on that would break that. (I've 
never amended a -native package but I use the same trick for other 
recipes where there are multiple .bb files are in the same OE directory.)

Mike

Re: A new bitbake extension: .bbappend files

[Chris Larson]

On Tue, Jul 20, 2010 at 8:52 AM, Michael Smith <msmith@cbnco.com> wrote:

> Tom Rini wrote:
>
>> It's also needed, in some cases, to lock down your changes via OVERRIDES
>> that will only match what you want to work on.  For example, we're using a
>> snapshot with an older python, so amend.inc in our python directory looks
>> like:
>>
>> SRC_URI_append_pn-python-native =
>> "file://python-2.6-r73342.patch;patch=1;pnum=0"
>> PARALLEL_MAKE_pn-python-native = ""
>> PR_append_pn-python-native .= ".1"
>>
>> So that we only change python-native and not all of the other python
>> recipes.
>>
>
> I think you could get the same effect by putting your amend.inc in a
> python-native subdirectory under your overlay recipes/python directory.
>
> Maybe there's some native magic going on that would break that. (I've never
> amended a -native package but I use the same trick for other recipes where
> there are multiple .bb files are in the same OE directory.)


Well, that works for -native or -cross, but there's no -target, so you have
to use a pn- override to modify the target recipe but *not* native and
cross.  If you put it in python/, it applies to all of them.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Frans Meulenbroeks]

2010/7/20 Chris Larson <clarson@kergoth.com>

> On Tue, Jul 20, 2010 at 8:52 AM, Michael Smith <msmith@cbnco.com> wrote:
>
> > Tom Rini wrote:
> >
> >> It's also needed, in some cases, to lock down your changes via OVERRIDES
> >> that will only match what you want to work on.  For example, we're using
> a
> >> snapshot with an older python, so amend.inc in our python directory
> looks
> >> like:
> >>
> >> SRC_URI_append_pn-python-native =
> >> "file://python-2.6-r73342.patch;patch=1;pnum=0"
> >> PARALLEL_MAKE_pn-python-native = ""
> >> PR_append_pn-python-native .= ".1"
> >>
> >> So that we only change python-native and not all of the other python
> >> recipes.
> >>
> >
> > I think you could get the same effect by putting your amend.inc in a
> > python-native subdirectory under your overlay recipes/python directory.
> >
> > Maybe there's some native magic going on that would break that. (I've
> never
> > amended a -native package but I use the same trick for other recipes
> where
> > there are multiple .bb files are in the same OE directory.)
>
>
> Well, that works for -native or -cross, but there's no -target, so you have
> to use a pn- override to modify the target recipe but *not* native and
> cross.  If you put it in python/, it applies to all of them.
>

Should we have/create a -native ? Seems a logical extension.
(but don't know how to code it)

Frans.

> --
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>

Re: A new bitbake extension: .bbappend files

[Chris Larson]

On Tue, Jul 20, 2010 at 9:29 AM, Frans Meulenbroeks <
fransmeulenbroeks@gmail.com> wrote:

> 2010/7/20 Chris Larson <clarson@kergoth.com>
>
> > On Tue, Jul 20, 2010 at 8:52 AM, Michael Smith <msmith@cbnco.com> wrote:
> >
> > > Tom Rini wrote:
> > >
> > >> It's also needed, in some cases, to lock down your changes via
> OVERRIDES
> > >> that will only match what you want to work on.  For example, we're
> using
> > a
> > >> snapshot with an older python, so amend.inc in our python directory
> > looks
> > >> like:
> > >>
> > >> SRC_URI_append_pn-python-native =
> > >> "file://python-2.6-r73342.patch;patch=1;pnum=0"
> > >> PARALLEL_MAKE_pn-python-native = ""
> > >> PR_append_pn-python-native .= ".1"
> > >>
> > >> So that we only change python-native and not all of the other python
> > >> recipes.
> > >>
> > >
> > > I think you could get the same effect by putting your amend.inc in a
> > > python-native subdirectory under your overlay recipes/python directory.
> > >
> > > Maybe there's some native magic going on that would break that. (I've
> > never
> > > amended a -native package but I use the same trick for other recipes
> > where
> > > there are multiple .bb files are in the same OE directory.)
> >
> >
> > Well, that works for -native or -cross, but there's no -target, so you
> have
> > to use a pn- override to modify the target recipe but *not* native and
> > cross.  If you put it in python/, it applies to all of them.
> >
>
> Should we have/create a -native ? Seems a logical extension.
> (but don't know how to code it)


I assume you mean -target.  You could easily enough add it to FILESPATHPKG
conditionally based on inherits_class() of
native/cross/nativesdk/crosssdk/etc.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Frans Meulenbroeks]

Wrt the bbappend / amend.inc discussion:

I was wondering how version management would work.
I can imagine that if I change something in a bbappend file (or amend.inc)
file, I would like to have something like PR that I could bump.
In my case these recipes live in an overlay which is also used by
co-workers. I would like to be able to trigger a rebuild when they get a
modified .bbappend file.

Frans

Re: A new bitbake extension: .bbappend files

[Richard Purdie]

On Tue, 2010-08-03 at 12:04 +0200, Frans Meulenbroeks wrote:
> Wrt the bbappend / amend.inc discussion:
> 
> I was wondering how version management would work.
> I can imagine that if I change something in a bbappend file (or amend.inc)
> file, I would like to have something like PR that I could bump.
> In my case these recipes live in an overlay which is also used by
> co-workers. I would like to be able to trigger a rebuild when they get a
> modified .bbappend file.

You could use something like:

PR := "${PR}.1"

Cheers,

Richard

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Richard Purdie]

On Fri, 2010-07-16 at 15:25 +0100, Richard Purdie wrote:
> Whilst our layers mechanism, is great it does have a drawback which has
> bugged me for a while. If you have a recipe like pointercal which has
> machine specific information in it and you have your new machine code in
> a layer, how do you add a pointercal file for your machine?
> 
> Answer is you copy the whole pointercal recipe and files into your
> layer, then add the single file for your machine. To me this is ugly,
> ugly, ugly. We hate code duplication and as soon as you create two
> copies of the same information, we've failed.
> 
> So how could we do this better? Somehow we need to say that a given
> directory X has some information which should be merged with the
> original recipe. I've thought through several different ways of doing
> this and the best solution I found was "bbappend".
> 
> The idea is that if bitbake finds any X.bbappend files, when it loads
> X.bb, it will also include these files after it parses the base .bb file
> (but before finalise and the anonymous methods run). This means that
> the .bbappend file can poke around and do whatever it might want to the
> recipe to customise it.
> 
> I went ahead and tried it out as its quite simple to code this in
> bitbake. I liked the result enough I've already merged this into Poky:
> 
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> 
> I'm proposing to push it to bitbake master if there are no serious
> objections.

Whilst I think there were some initial concerns the consensus seemed to
be that this was a good thing to have in bitbake, its something that has
been on the feature wishlist since day one so I've merged the patch into
bitbake master.

Cheers,

Richard

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Chris Larson]

On Tue, Aug 3, 2010 at 6:26 AM, Richard Purdie <rpurdie@rpsys.net> wrote:

> On Fri, 2010-07-16 at 15:25 +0100, Richard Purdie wrote:
> > Whilst our layers mechanism, is great it does have a drawback which has
> > bugged me for a while. If you have a recipe like pointercal which has
> > machine specific information in it and you have your new machine code in
> > a layer, how do you add a pointercal file for your machine?
> >
> > Answer is you copy the whole pointercal recipe and files into your
> > layer, then add the single file for your machine. To me this is ugly,
> > ugly, ugly. We hate code duplication and as soon as you create two
> > copies of the same information, we've failed.
> >
> > So how could we do this better? Somehow we need to say that a given
> > directory X has some information which should be merged with the
> > original recipe. I've thought through several different ways of doing
> > this and the best solution I found was "bbappend".
> >
> > The idea is that if bitbake finds any X.bbappend files, when it loads
> > X.bb, it will also include these files after it parses the base .bb file
> > (but before finalise and the anonymous methods run). This means that
> > the .bbappend file can poke around and do whatever it might want to the
> > recipe to customise it.
> >
> > I went ahead and tried it out as its quite simple to code this in
> > bitbake. I liked the result enough I've already merged this into Poky:
> >
> >
> http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> >
> > I'm proposing to push it to bitbake master if there are no serious
> > objections.
>
> Whilst I think there were some initial concerns the consensus seemed to
> be that this was a good thing to have in bitbake, its something that has
> been on the feature wishlist since day one so I've merged the patch into
> bitbake master.
>

Glad to hear it, nice work putting it together.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: A new bitbake extension: .bbappend files

[Khem Raj]

On (03/08/10 12:04), Frans Meulenbroeks wrote:
> Wrt the bbappend / amend.inc discussion:
> 
> I was wondering how version management would work.
> I can imagine that if I change something in a bbappend file (or amend.inc)
> file, I would like to have something like PR that I could bump.
> In my case these recipes live in an overlay which is also used by
> co-workers. I would like to be able to trigger a rebuild when they get a
> modified .bbappend file.

may be use INC_PR 
> 
> Frans
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Khem Raj]

On (03/08/10 14:26), Richard Purdie wrote:
> On Fri, 2010-07-16 at 15:25 +0100, Richard Purdie wrote:
> > Whilst our layers mechanism, is great it does have a drawback which has
> > bugged me for a while. If you have a recipe like pointercal which has
> > machine specific information in it and you have your new machine code in
> > a layer, how do you add a pointercal file for your machine?
> > 
> > Answer is you copy the whole pointercal recipe and files into your
> > layer, then add the single file for your machine. To me this is ugly,
> > ugly, ugly. We hate code duplication and as soon as you create two
> > copies of the same information, we've failed.
> > 
> > So how could we do this better? Somehow we need to say that a given
> > directory X has some information which should be merged with the
> > original recipe. I've thought through several different ways of doing
> > this and the best solution I found was "bbappend".
> > 
> > The idea is that if bitbake finds any X.bbappend files, when it loads
> > X.bb, it will also include these files after it parses the base .bb file
> > (but before finalise and the anonymous methods run). This means that
> > the .bbappend file can poke around and do whatever it might want to the
> > recipe to customise it.
> > 
> > I went ahead and tried it out as its quite simple to code this in
> > bitbake. I liked the result enough I've already merged this into Poky:
> > 
> > http://git.pokylinux.org/cgit.cgi/poky/commit/?id=63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660
> > 
> > I'm proposing to push it to bitbake master if there are no serious
> > objections.
> 
> Whilst I think there were some initial concerns the consensus seemed to
> be that this was a good thing to have in bitbake, its something that has
> been on the feature wishlist since day one so I've merged the patch into
> bitbake master.

Now this fails.

$ bitbake -c clean -b virtual:native:/home/kraj/work/oe/openembedded/recipes/zlib/zlib_1.2.3.bb
Command execution failed: Traceback (most recent call last):
  File "/home/kraj/work/oe/bitbake/lib/bb/command.py", line 88, in
runAsyncCommand
    commandmethod(self.cmds_async, self, options)
  File "/home/kraj/work/oe/bitbake/lib/bb/command.py", line 174, in
buildFile
    command.cooker.buildFile(bfile, task)
  File "/home/kraj/work/oe/bitbake/lib/bb/cooker.py", line 640, in
buildFile
    the_data = self.bb_cache.loadDataFull(fn, self.get_file_appends(fn),
self.configuration.data)
  File "/home/kraj/work/oe/bitbake/lib/bb/cooker.py", line 897, in
get_file_appends
    if f in self.appendlist:
AttributeError: BBCooker instance has no attribute 'appendlist'


> 
> Cheers,
> 
> Richard
> 
> _______________________________________________
> Bitbake-dev mailing list
> Bitbake-dev@lists.berlios.de
> https://lists.berlios.de/mailman/listinfo/bitbake-dev

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Richard Purdie]

On Tue, 2010-08-03 at 21:09 -0700, Khem Raj wrote:
> On (03/08/10 14:26), Richard Purdie wrote:
> > Whilst I think there were some initial concerns the consensus seemed to
> > be that this was a good thing to have in bitbake, its something that has
> > been on the feature wishlist since day one so I've merged the patch into
> > bitbake master.
> 
> Now this fails.
> 
> $ bitbake -c clean -b virtual:native:/home/kraj/work/oe/openembedded/recipes/zlib/zlib_1.2.3.bb
> Command execution failed: Traceback (most recent call last):
>   File "/home/kraj/work/oe/bitbake/lib/bb/command.py", line 88, in
> runAsyncCommand
>     commandmethod(self.cmds_async, self, options)
>   File "/home/kraj/work/oe/bitbake/lib/bb/command.py", line 174, in
> buildFile
>     command.cooker.buildFile(bfile, task)
>   File "/home/kraj/work/oe/bitbake/lib/bb/cooker.py", line 640, in
> buildFile
>     the_data = self.bb_cache.loadDataFull(fn, self.get_file_appends(fn),
> self.configuration.data)
>   File "/home/kraj/work/oe/bitbake/lib/bb/cooker.py", line 897, in
> get_file_appends
>     if f in self.appendlist:
> AttributeError: BBCooker instance has no attribute 'appendlist'

Sorry, I'd tested with partial expressions in the -b option, not full
paths. I've pushed a fix.

Cheers,

Richard

Re: [Bitbake-dev] A new bitbake extension: .bbappend files

[Khem Raj]

On Wed, Aug 4, 2010 at 7:40 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
> On Tue, 2010-08-03 at 21:09 -0700, Khem Raj wrote:
>> On (03/08/10 14:26), Richard Purdie wrote:
>> > Whilst I think there were some initial concerns the consensus seemed to
>> > be that this was a good thing to have in bitbake, its something that has
>> > been on the feature wishlist since day one so I've merged the patch into
>> > bitbake master.
>>
>> Now this fails.
>>
>> $ bitbake -c clean -b virtual:native:/home/kraj/work/oe/openembedded/recipes/zlib/zlib_1.2.3.bb
>> Command execution failed: Traceback (most recent call last):
>>   File "/home/kraj/work/oe/bitbake/lib/bb/command.py", line 88, in
>> runAsyncCommand
>>     commandmethod(self.cmds_async, self, options)
>>   File "/home/kraj/work/oe/bitbake/lib/bb/command.py", line 174, in
>> buildFile
>>     command.cooker.buildFile(bfile, task)
>>   File "/home/kraj/work/oe/bitbake/lib/bb/cooker.py", line 640, in
>> buildFile
>>     the_data = self.bb_cache.loadDataFull(fn, self.get_file_appends(fn),
>> self.configuration.data)
>>   File "/home/kraj/work/oe/bitbake/lib/bb/cooker.py", line 897, in
>> get_file_appends
>>     if f in self.appendlist:
>> AttributeError: BBCooker instance has no attribute 'appendlist'
>
> Sorry, I'd tested with partial expressions in the -b option, not full
> paths. I've pushed a fix.

thanks works well now :)

>
> Cheers,
>
> Richard
>
>