* Layer Priority with Wildcard .bbappend Files
@ 2014-11-24 20:22 Stevens, Nick
2014-11-24 21:25 ` Bob Cochran
0 siblings, 1 reply; 7+ messages in thread
From: Stevens, Nick @ 2014-11-24 20:22 UTC (permalink / raw)
To: yocto
I think I've encountered a bug with how multiple bbappend files are processed when one of the bbappends contains a filename wildcard, but I want to make sure there's not something I'm missing before filing a bug report.
I have a BSP layer and a customization layer that are based on the OE/poky Daisy release. The output of `bitbake-layers show-layers` looks like:
layer path priority
=====================================================
meta poky/meta 5
meta-yocto poky/meta-yocto 5
meta-yocto-bsp poky/meta-yocto-bsp 5
...ellided...
meta-bsp meta-bsp 6
meta-custom meta-custom 8
In meta-bsp there is a bbappend for base-files named base-files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc - pretty straightforward:
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI += "file://sysctl.conf"
do_install_append() {
install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
}
Now what I want to do is add a file called base-files_%.bbappend to meta-custom. It has its own version of sysctl.conf, and the recipe looks like this:
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
Here's where things get weird. If I run `bitbake -e base-files` and pull out the section for FILESEXTRAPATHS, this is what I get (note that I've stripped out the huge absolute paths to make this easier to read):
# $FILESEXTRAPATHS [5 operations]
# set poky/meta/conf/documentation.conf:172
# [doc] "Extends the search path the OpenEmbedded build system uses when looking for files and patches as it processes recipes and append files."
# _prepend meta-custom/recipes-core/base-files/base-files_%.bbappend:6
# "meta-custom/recipes-core/base-files/base-files:"
# _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
# "meta-bsp/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-custom/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
# computed:
# "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS, even though meta-custom has a higher priority! I verified this by building an image - the sysctl.conf that ends up in the final image is the one from meta-bsp, not the one from meta-custom. But if I switch the name of base-files_%.bbappend in meta-custom to base-files_3.0.14.bbappend, this is what I get:
# $FILESEXTRAPATHS [5 operations]
# set poky/meta/conf/documentation.conf:172
# [doc] "Extends the search path the OpenEmbedded build system uses when looking for files and patches as it processes recipes and append files."
# _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
# "meta-bsp/recipes-core/base-files/base-files:"
# _prepend meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6
# "meta-custom/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-bsp/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
# computed:
# "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
And now the sysctl.conf file is being pulled from meta-custom.
Is there something I'm missing, like some sort of decreased priorty for bbappend files with a wildcard? Or is this a bug?
Thanks for any help!
Nick Stevens
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Layer Priority with Wildcard .bbappend Files
2014-11-24 20:22 Layer Priority with Wildcard .bbappend Files Stevens, Nick
@ 2014-11-24 21:25 ` Bob Cochran
2014-11-25 11:12 ` Paul Eggleton
0 siblings, 1 reply; 7+ messages in thread
From: Bob Cochran @ 2014-11-24 21:25 UTC (permalink / raw)
To: Stevens, Nick, yocto
On 11/24/2014 03:22 PM, Stevens, Nick wrote:
> I think I've encountered a bug with how multiple bbappend files are processed when one of the bbappends contains a filename wildcard, but I want to make sure there's not something I'm missing before filing a bug report.
>
> I have a BSP layer and a customization layer that are based on the OE/poky Daisy release. The output of `bitbake-layers show-layers` looks like:
> layer path priority
> =====================================================
> meta poky/meta 5
> meta-yocto poky/meta-yocto 5
> meta-yocto-bsp poky/meta-yocto-bsp 5
> ...ellided...
> meta-bsp meta-bsp 6
> meta-custom meta-custom 8
>
> In meta-bsp there is a bbappend for base-files named base-files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc - pretty straightforward:
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
> SRC_URI += "file://sysctl.conf"
> do_install_append() {
> install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
> }
>
> Now what I want to do is add a file called base-files_%.bbappend to meta-custom. It has its own version of sysctl.conf, and the recipe looks like this:
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>
> Here's where things get weird. If I run `bitbake -e base-files` and pull out the section for FILESEXTRAPATHS, this is what I get (note that I've stripped out the huge absolute paths to make this easier to read):
> # $FILESEXTRAPATHS [5 operations]
> # set poky/meta/conf/documentation.conf:172
> # [doc] "Extends the search path the OpenEmbedded build system uses when looking for files and patches as it processes recipes and append files."
> # _prepend meta-custom/recipes-core/base-files/base-files_%.bbappend:6
> # "meta-custom/recipes-core/base-files/base-files:"
> # _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
> # "meta-bsp/recipes-core/base-files/base-files:"
> # set data_smart.py:432 [finalize]
> # "meta-custom/recipes-core/base-files/base-files:"
> # set data_smart.py:432 [finalize]
> # "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
> # computed:
> # "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
> FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
>
> For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS, even though meta-custom has a higher priority!
I also have some questions about how FILESEXTRAPATHS is supposed to work
with append files and layer priorities.
Does a higher layer priority mean that the FILESEXTRAPATHS operation
should occur first? If this is the case, then a lower priority layer
prepend will appear to the left of the higher layer prepend since it
runs later, and this will probably not provide the result you want.
The prepend and layer logic seems messy to me. I would like to have a
way to write my custom layer and specify that what I include in my
SRC_URI in each recipe is definitive and can not be overwritten.
Suggestions on how to best accomplish this will be greatly appreciated.
I have been trying to accomplish this with FILESOVERRIDES, but this
logic seems counterintuitive since DISTROOVERRIDES take precedence over
MACHINEOVERRIDES in building the file search path during the unpack task.
It seems to me that the file search path should be built using
FILESOVERRIDES from left to right: TRANSLATED_TARGET_ARCH,
MACHINEOVERRIDES, and then DISTROOVERRIDES (specific to generic), but
this isn't what I'm seeing. Still investigating how it's all put
together...
I verified this by building an image - the sysctl.conf that ends up in
the final image is the one from meta-bsp, not the one from meta-custom.
But if I switch the name of base-files_%.bbappend in meta-custom to
base-files_3.0.14.bbappend, this is what I get:
> # $FILESEXTRAPATHS [5 operations]
> # set poky/meta/conf/documentation.conf:172
> # [doc] "Extends the search path the OpenEmbedded build system uses when looking for files and patches as it processes recipes and append files."
> # _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
> # "meta-bsp/recipes-core/base-files/base-files:"
> # _prepend meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6
> # "meta-custom/recipes-core/base-files/base-files:"
> # set data_smart.py:432 [finalize]
> # "meta-bsp/recipes-core/base-files/base-files:"
> # set data_smart.py:432 [finalize]
> # "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
> # computed:
> # "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
> FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
>
> And now the sysctl.conf file is being pulled from meta-custom.
>
> Is there something I'm missing, like some sort of decreased priorty for bbappend files with a wildcard? Or is this a bug?
>
> Thanks for any help!
>
> Nick Stevens
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Layer Priority with Wildcard .bbappend Files
2014-11-24 21:25 ` Bob Cochran
@ 2014-11-25 11:12 ` Paul Eggleton
2014-11-25 18:04 ` Stevens, Nick
2014-11-25 20:38 ` Bob Cochran
0 siblings, 2 replies; 7+ messages in thread
From: Paul Eggleton @ 2014-11-25 11:12 UTC (permalink / raw)
To: Bob Cochran, Stevens, Nick; +Cc: yocto
Hi Bob / Nick,
On Monday 24 November 2014 16:25:58 Bob Cochran wrote:
> On 11/24/2014 03:22 PM, Stevens, Nick wrote:
> > I think I've encountered a bug with how multiple bbappend files are
> > processed when one of the bbappends contains a filename wildcard, but I
> > want to make sure there's not something I'm missing before filing a bug
> > report.>
> > I have a BSP layer and a customization layer that are based on the OE/poky
> > Daisy release. The output of `bitbake-layers show-layers` looks like:
> > layer path priority
> > =====================================================
> > meta poky/meta 5
> > meta-yocto poky/meta-yocto 5
> > meta-yocto-bsp poky/meta-yocto-bsp 5
> > ...ellided...
> > meta-bsp meta-bsp 6
> > meta-custom meta-custom 8
> >
> > In meta-bsp there is a bbappend for base-files named base-
> > files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc -
> > pretty straightforward:
> > FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
> > SRC_URI += "file://sysctl.conf"
> > do_install_append() {
> >
> > install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
> >
> > }
> >
> > Now what I want to do is add a file called base-files_%.bbappend to meta-
> > custom. It has its own version of sysctl.conf, and the recipe looks like
> > this:
> > FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
> >
> > Here's where things get weird. If I run `bitbake -e base-files` and pull
> > out the section for FILESEXTRAPATHS, this is what I get (note that I've
> > stripped out the huge absolute paths to make this easier to read):
> > # $FILESEXTRAPATHS [5 operations]
> > # set poky/meta/conf/documentation.conf:172
> > # [doc] "Extends the search path the OpenEmbedded build system
> > uses when looking for files and patches as it processes recipes and
> > append files." # _prepend
> > meta-custom/recipes-core/base-files/base-files_%.bbappend:6 #
> > "meta-custom/recipes-core/base-files/base-files:"
> > # _prepend
> > meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
> > # "meta-bsp/recipes-core/base-files/base-files:"
> > # set data_smart.py:432 [finalize]
> > # "meta-custom/recipes-core/base-files/base-files:"
> > # set data_smart.py:432 [finalize]
> > #
> > "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-cor
> > e/base-files/base-files:" # computed:
> > #
> > "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-cor
> > e/base-files/base-files:"
> > FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-cu
> > stom/recipes-core/base-files/base-files:">
> > For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS,
> > even though meta-custom has a higher priority!
>
> I also have some questions about how FILESEXTRAPATHS is supposed to work
> with append files and layer priorities.
>
> Does a higher layer priority mean that the FILESEXTRAPATHS operation
> should occur first?
No. bbappends are parsed in ascending layer priority order. It wouldn't make
sense to do it in the reverse - you want the higher priority layer's settings
to take precedence
> If this is the case, then a lower priority layer prepend will appear to the
> left of the higher layer prepend since it runs later, and this will probably
> not provide the result you want.
The highest priority layer's bbappend will prepend last, thus anything it
prepends will be first in the list.
> The prepend and layer logic seems messy to me. I would like to have a
> way to write my custom layer and specify that what I include in my
> SRC_URI in each recipe is definitive and can not be overwritten.
> Suggestions on how to best accomplish this will be greatly appreciated.
I think this is pretty much already the case. Is the problem that you can't
follow the current behaviour, or that you don't completely trust the layers
you are pulling in?
> I have been trying to accomplish this with FILESOVERRIDES, but this
> logic seems counterintuitive since DISTROOVERRIDES take precedence over
> MACHINEOVERRIDES in building the file search path during the unpack task.
>
> It seems to me that the file search path should be built using
> FILESOVERRIDES from left to right: TRANSLATED_TARGET_ARCH,
> MACHINEOVERRIDES, and then DISTROOVERRIDES (specific to generic), but
> this isn't what I'm seeing. Still investigating how it's all put
> together...
>
>
> I verified this by building an image - the sysctl.conf that ends up in
> the final image is the one from meta-bsp, not the one from meta-custom.
> But if I switch the name of base-files_%.bbappend in meta-custom to
> base-files_3.0.14.bbappend, this is what I get:
> > # $FILESEXTRAPATHS [5 operations]
> > # set poky/meta/conf/documentation.conf:172
> > # [doc] "Extends the search path the OpenEmbedded build system
> > uses when looking for files and patches as it processes recipes and
> > append files." # _prepend
> > meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3 #
> > "meta-bsp/recipes-core/base-files/base-files:"
> > # _prepend
> > meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6 #
> > "meta-custom/recipes-core/base-files/base-files:"
> > # set data_smart.py:432 [finalize]
> > # "meta-bsp/recipes-core/base-files/base-files:"
> > # set data_smart.py:432 [finalize]
> > #
> > "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-cor
> > e/base-files/base-files:" # computed:
> > #
> > "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-cor
> > e/base-files/base-files:"
> > FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta
> > -bsp/recipes-core/base-files/base-files:">
> > And now the sysctl.conf file is being pulled from meta-custom.
That absolutely should not happen. If you change it back does it revert to the
version from meta-bsp? The wildcarding should not change the order in which
the files are applied. The only corner case I can think of would be if the
layer priority values are the same number - this isn't the case is it?
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Layer Priority with Wildcard .bbappend Files
2014-11-25 11:12 ` Paul Eggleton
@ 2014-11-25 18:04 ` Stevens, Nick
2014-11-25 20:38 ` Bob Cochran
1 sibling, 0 replies; 7+ messages in thread
From: Stevens, Nick @ 2014-11-25 18:04 UTC (permalink / raw)
To: Paul Eggleton, Bob Cochran; +Cc: yocto
Hi Paul,
> > I verified this by building an image - the sysctl.conf that ends up in
> > the final image is the one from meta-bsp, not the one from meta-custom.
> > But if I switch the name of base-files_%.bbappend in meta-custom to
> > base-files_3.0.14.bbappend, this is what I get:
> > > # $FILESEXTRAPATHS [5 operations]
> > > # set poky/meta/conf/documentation.conf:172
> > > # [doc] "Extends the search path the OpenEmbedded build
> system
> > > uses when looking for files and patches as it processes recipes
> and
> > > append files." # _prepend
> > > meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3 #
> > > "meta-bsp/recipes-core/base-files/base-files:"
> > > # _prepend
> > > meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6
> #
> > > "meta-custom/recipes-core/base-files/base-files:"
> > > # set data_smart.py:432 [finalize]
> > > # "meta-bsp/recipes-core/base-files/base-files:"
> > > # set data_smart.py:432 [finalize]
> > > #
> > > "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-
> cor
> > > e/base-files/base-files:" # computed:
> > > #
> > > "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-
> cor
> > > e/base-files/base-files:"
> > > FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-
> files:meta
> > > -bsp/recipes-core/base-files/base-files:">
> > > And now the sysctl.conf file is being pulled from meta-custom.
>
> That absolutely should not happen. If you change it back does it revert to
> the version from meta-bsp? The wildcarding should not change the order in
> which the files are applied. The only corner case I can think of would be
> if the layer priority values are the same number - this isn't the case is
> it?
The behavior does reappear if I change the file name back to a wildcard. And no, the layers are different priorities - meta-custom is priority 8 and meta-bsp is priority 6.
Definitely sounds like there's some unexpected behavior here. I am going to dig in some more and see if I can find a limited test case that reproduces the issue, and then I'll get a bug report opened.
Thanks!
Nick
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Layer Priority with Wildcard .bbappend Files
2014-11-25 11:12 ` Paul Eggleton
2014-11-25 18:04 ` Stevens, Nick
@ 2014-11-25 20:38 ` Bob Cochran
1 sibling, 0 replies; 7+ messages in thread
From: Bob Cochran @ 2014-11-25 20:38 UTC (permalink / raw)
To: Paul Eggleton, Stevens, Nick; +Cc: yocto
On 11/25/2014 06:12 AM, Paul Eggleton wrote:
> Hi Bob / Nick,
>
> On Monday 24 November 2014 16:25:58 Bob Cochran wrote:
>> On 11/24/2014 03:22 PM, Stevens, Nick wrote:
>>> I think I've encountered a bug with how multiple bbappend files are
>>> processed when one of the bbappends contains a filename wildcard, but I
>>> want to make sure there's not something I'm missing before filing a bug
>>> report.>
>>> I have a BSP layer and a customization layer that are based on the OE/poky
>>> Daisy release. The output of `bitbake-layers show-layers` looks like:
>>> layer path priority
>>> =====================================================
>>> meta poky/meta 5
>>> meta-yocto poky/meta-yocto 5
>>> meta-yocto-bsp poky/meta-yocto-bsp 5
>>> ...ellided...
>>> meta-bsp meta-bsp 6
>>> meta-custom meta-custom 8
>>>
>>> In meta-bsp there is a bbappend for base-files named base-
>>> files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc -
>>> pretty straightforward:
>>> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>>> SRC_URI += "file://sysctl.conf"
>>> do_install_append() {
>>>
>>> install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
>>>
>>> }
>>>
>>> Now what I want to do is add a file called base-files_%.bbappend to meta-
>>> custom. It has its own version of sysctl.conf, and the recipe looks like
>>> this:
>>> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>>>
>>> Here's where things get weird. If I run `bitbake -e base-files` and pull
>>> out the section for FILESEXTRAPATHS, this is what I get (note that I've
>>> stripped out the huge absolute paths to make this easier to read):
>>> # $FILESEXTRAPATHS [5 operations]
>>> # set poky/meta/conf/documentation.conf:172
>>> # [doc] "Extends the search path the OpenEmbedded build system
>>> uses when looking for files and patches as it processes recipes and
>>> append files." # _prepend
>>> meta-custom/recipes-core/base-files/base-files_%.bbappend:6 #
>>> "meta-custom/recipes-core/base-files/base-files:"
>>> # _prepend
>>> meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
>>> # "meta-bsp/recipes-core/base-files/base-files:"
>>> # set data_smart.py:432 [finalize]
>>> # "meta-custom/recipes-core/base-files/base-files:"
>>> # set data_smart.py:432 [finalize]
>>> #
>>> "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-cor
>>> e/base-files/base-files:" # computed:
>>> #
>>> "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-cor
>>> e/base-files/base-files:"
>>> FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-cu
>>> stom/recipes-core/base-files/base-files:">
>>> For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS,
>>> even though meta-custom has a higher priority!
>>
>> I also have some questions about how FILESEXTRAPATHS is supposed to work
>> with append files and layer priorities.
>>
>> Does a higher layer priority mean that the FILESEXTRAPATHS operation
>> should occur first?
>
> No. bbappends are parsed in ascending layer priority order. It wouldn't make
> sense to do it in the reverse - you want the higher priority layer's settings
> to take precedence
>
>> If this is the case, then a lower priority layer prepend will appear to the
>> left of the higher layer prepend since it runs later, and this will probably
>> not provide the result you want.
>
> The highest priority layer's bbappend will prepend last, thus anything it
> prepends will be first in the list.
>
>> The prepend and layer logic seems messy to me. I would like to have a
>> way to write my custom layer and specify that what I include in my
>> SRC_URI in each recipe is definitive and can not be overwritten.
>> Suggestions on how to best accomplish this will be greatly appreciated.
>
> I think this is pretty much already the case. Is the problem that you can't
> follow the current behaviour, or that you don't completely trust the layers
> you are pulling in?
Thanks Paul. Something is amiss when I try to prepend my recipes. I
suspect it has something to do with OVERRIDE interactions. I'm
digging and will report later.
>
>> I have been trying to accomplish this with FILESOVERRIDES, but this
>> logic seems counterintuitive since DISTROOVERRIDES take precedence over
>> MACHINEOVERRIDES in building the file search path during the unpack task.
>>
>> It seems to me that the file search path should be built using
>> FILESOVERRIDES from left to right: TRANSLATED_TARGET_ARCH,
>> MACHINEOVERRIDES, and then DISTROOVERRIDES (specific to generic), but
>> this isn't what I'm seeing. Still investigating how it's all put
>> together...
>>
>>
>> I verified this by building an image - the sysctl.conf that ends up in
>> the final image is the one from meta-bsp, not the one from meta-custom.
>> But if I switch the name of base-files_%.bbappend in meta-custom to
>> base-files_3.0.14.bbappend, this is what I get:
>>> # $FILESEXTRAPATHS [5 operations]
>>> # set poky/meta/conf/documentation.conf:172
>>> # [doc] "Extends the search path the OpenEmbedded build system
>>> uses when looking for files and patches as it processes recipes and
>>> append files." # _prepend
>>> meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3 #
>>> "meta-bsp/recipes-core/base-files/base-files:"
>>> # _prepend
>>> meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6 #
>>> "meta-custom/recipes-core/base-files/base-files:"
>>> # set data_smart.py:432 [finalize]
>>> # "meta-bsp/recipes-core/base-files/base-files:"
>>> # set data_smart.py:432 [finalize]
>>> #
>>> "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-cor
>>> e/base-files/base-files:" # computed:
>>> #
>>> "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-cor
>>> e/base-files/base-files:"
>>> FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta
>>> -bsp/recipes-core/base-files/base-files:">
>>> And now the sysctl.conf file is being pulled from meta-custom.
>
> That absolutely should not happen. If you change it back does it revert to the
> version from meta-bsp? The wildcarding should not change the order in which
> the files are applied. The only corner case I can think of would be if the
> layer priority values are the same number - this isn't the case is it?
>
> Cheers,
> Paul
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Layer Priority with Wildcard .bbappend Files
2015-05-16 11:10 Christian Magnusson
@ 2015-05-16 18:42 ` Christopher Larson
0 siblings, 0 replies; 7+ messages in thread
From: Christopher Larson @ 2015-05-16 18:42 UTC (permalink / raw)
To: mag, yocto
[-- Attachment #1: Type: text/plain, Size: 9568 bytes --]
Richard fixed this in master and fido after I opened a Yocto bug about it.
I don't recall the commit offhand, but you may be able to cherry-pick it.
On Sat, May 16, 2015 at 4:16 AM Christian Magnusson <mag@mag.cx> wrote:
> I also stumbled into this problem.
>
> We are using a BSP from Phytec. It’s using poky and Phytec made a simple distribution in meta-yougurt.
>
> Now I want to customize it, and made my own meta-platform to overwrite files from the other layers. (prio 100)
>
>
>
> # bitbake-layers show-recipes
>
> init-ifupdown:
>
> meta 1.0
>
>
>
> # bitbake-layers show-layers
>
> layer path priority
>
> ==========================================================================
>
> meta /home/s22463/yocto/sources/poky/meta 5
>
> meta-yocto /home/s22463/yocto/sources/poky/meta-yocto 5
>
> meta-phytec /home/s22463/yocto/sources/meta-phytec 20
>
> meta-phyam335x /home/s22463/yocto/sources/meta-phytec/meta-phyam335x 20
>
> meta-yogurt /home/s22463/yocto/sources/meta-yogurt 10
>
> meta-oe /home/s22463/yocto/sources/meta-openembedded/meta-oe 6
>
> meta-networking /home/s22463/yocto/sources/meta-openembedded/meta-networking 5
>
> meta-python /home/s22463/yocto/sources/meta-openembedded/meta-python 7
>
> meta-systemd /home/s22463/yocto/sources/meta-openembedded/meta-systemd 7
>
> meta-qt5 /home/s22463/yocto/sources/meta-qt5 7
>
> meta-platform /home/s22463/yocto/sources/meta-platform 100
>
> meta-webserver /home/s22463/yocto/sources/meta-openembedded/meta-webserver 6
>
>
>
> # bitbake-layers show-appends
>
> init-ifupdown_1.0.bb:
>
> /home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend
>
> /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_%.bbappend
>
>
>
> Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf doesn’t affect the order either.
>
>
>
>
>
> If I rename
>
> /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_%.bbappend -> init-ifupdown_1.0.bbappend
>
> the priority works, and my files in meta-platform are installed last, as I want to overwrite all other files.
>
>
>
> # mv /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_%.bbappend /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend
>
> # bitbake-layers show-appends
>
> init-ifupdown_1.0.bb:
>
> /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend
>
> /home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend
>
>
>
> The installation order works now, but I would rather avoid to patch the other layers.
>
> Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf now, will affect the order!, and I can install meta-plaform either before or after the meta-yougurt layer if I want.
>
>
>
> Does anyone know the reason to the faulty sort-order?
>
>
>
> /Christian Magnusson
>
>
>
>
>
>
>
> On 11/24/2014 03:22 PM, Stevens, Nick wrote:
>
> I think I've encountered a bug with how multiple bbappend files are processed
>
> when one of the bbappends contains a filename wildcard, but I want to make sure
>
> there's not something I'm missing before filing a bug report.
>
>
>
> I have a BSP layer and a customization layer that are based on the OE/poky
>
> Daisy release. The output of `bitbake-layers show-layers` looks like:
>
> layer path priority
>
> =====================================================
>
> meta poky/meta 5
>
> meta-yocto poky/meta-yocto 5
>
> meta-yocto-bsp poky/meta-yocto-bsp 5
>
> ...ellided...
>
> meta-bsp meta-bsp 6
>
> meta-custom meta-custom 8
>
>
>
> In meta-bsp there is a bbappend for base-files named
>
> base-files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc -
>
> pretty straightforward:
>
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>
> SRC_URI += "file://sysctl.conf"
>
> do_install_append() {
>
> install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
>
> }
>
>
>
> Now what I want to do is add a file called base-files_%.bbappend to
>
> meta-custom. It has its own version of sysctl.conf, and the recipe looks like
>
> this:
>
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
>
>
>
> Here's where things get weird. If I run `bitbake -e base-files` and pull out
>
> the section for FILESEXTRAPATHS, this is what I get (note that I've stripped
>
> out the huge absolute paths to make this easier to read):
>
> # $FILESEXTRAPATHS [5 operations]
>
> # set poky/meta/conf/documentation.conf:172
>
> # [doc] "Extends the search path the OpenEmbedded build system uses when
>
> looking for files and patches as it processes recipes and append files."
>
> # _prepend meta-custom/recipes-core/base-files/base-files_%.bbappend:6
>
> # "meta-custom/recipes-core/base-files/base-files:"
>
> # _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
>
> # "meta-bsp/recipes-core/base-files/base-files:"
>
> # set data_smart.py:432 [finalize]
>
> # "meta-custom/recipes-core/base-files/base-files:"
>
> # set data_smart.py:432 [finalize]
>
> #
>
> "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
>
> # computed:
>
> #
>
> "meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
>
>
>
> FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"
>
>
>
> For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS, even
>
> though meta-custom has a higher priority!
>
>
>
>
>
> I also have some questions about how FILESEXTRAPATHS is supposed to work with
> append files and layer priorities.
>
>
>
> Does a higher layer priority mean that the FILESEXTRAPATHS operation should
> occur first? If this is the case, then a lower priority layer prepend
> will appear to the left of the higher layer prepend since it runs later,
> and this will probably not provide the result you want.
>
>
>
> The prepend and layer logic seems messy to me. I would like to have a way
> to write my custom layer and specify that what I include in my SRC_URI in
> each recipe is definitive and can not be overwritten. Suggestions on how
> to best accomplish this will be greatly appreciated.
>
>
>
> I have been trying to accomplish this with FILESOVERRIDES, but this logic
> seems counterintuitive since DISTROOVERRIDES take precedence over MACHINEOVERRIDES
> in building the file search path during the unpack task.
>
>
>
> It seems to me that the file search path should be built using FILESOVERRIDES
> from left to right: TRANSLATED_TARGET_ARCH, MACHINEOVERRIDES, and then
> DISTROOVERRIDES (specific to generic), but this isn't what I'm seeing.
> Still investigating how it's all put together...
>
>
>
>
>
> I verified this by building an image - the sysctl.conf that ends up in the
> final image is the one from meta-bsp, not the one from meta-custom. But
> if I switch the name of base-files_%.bbappend in meta-custom to base-files_3.0.14.bbappend,
> this is what I get:
>
> # $FILESEXTRAPATHS [5 operations]
>
> # set poky/meta/conf/documentation.conf:172
>
> # [doc] "Extends the search path the OpenEmbedded build system uses when
>
> looking for files and patches as it processes recipes and append files."
>
> # _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
>
> # "meta-bsp/recipes-core/base-files/base-files:"
>
> # _prepend
>
> meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6
>
> # "meta-custom/recipes-core/base-files/base-files:"
>
> # set data_smart.py:432 [finalize]
>
> # "meta-bsp/recipes-core/base-files/base-files:"
>
> # set data_smart.py:432 [finalize]
>
> #
>
> "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
>
> # computed:
>
> #
>
> "meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
>
>
>
> FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"
>
>
>
> And now the sysctl.conf file is being pulled from meta-custom.
>
>
>
> Is there something I'm missing, like some sort of decreased priorty for
>
> bbappend files with a wildcard? Or is this a bug?
>
>
>
> Thanks for any help!
>
>
>
> Nick Stevens
>
>
>
>
>
> --
>
> _______________________________________________
>
> yocto mailing list
>
> yocto@yoctoproject.org
>
> https://lists.yoctoproject.org/listinfo/yocto
>
>
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
[-- Attachment #2: Type: text/html, Size: 38122 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Layer Priority with Wildcard .bbappend Files
@ 2015-05-16 11:10 Christian Magnusson
2015-05-16 18:42 ` Christopher Larson
0 siblings, 1 reply; 7+ messages in thread
From: Christian Magnusson @ 2015-05-16 11:10 UTC (permalink / raw)
To: yocto
[-- Attachment #1: Type: text/plain, Size: 8588 bytes --]
I also stumbled into this problem.
We are using a BSP from Phytec. It's using poky and Phytec made a simple
distribution in meta-yougurt.
Now I want to customize it, and made my own meta-platform to overwrite files
from the other layers. (prio 100)
# bitbake-layers show-recipes
init-ifupdown:
meta 1.0
# bitbake-layers show-layers
layer path priority
==========================================================================
meta /home/s22463/yocto/sources/poky/meta 5
meta-yocto /home/s22463/yocto/sources/poky/meta-yocto 5
meta-phytec /home/s22463/yocto/sources/meta-phytec 20
meta-phyam335x /home/s22463/yocto/sources/meta-phytec/meta-phyam335x
20
meta-yogurt /home/s22463/yocto/sources/meta-yogurt 10
meta-oe /home/s22463/yocto/sources/meta-openembedded/meta-oe
6
meta-networking
/home/s22463/yocto/sources/meta-openembedded/meta-networking 5
meta-python
/home/s22463/yocto/sources/meta-openembedded/meta-python 7
meta-systemd
/home/s22463/yocto/sources/meta-openembedded/meta-systemd 7
meta-qt5 /home/s22463/yocto/sources/meta-qt5 7
meta-platform /home/s22463/yocto/sources/meta-platform 100
meta-webserver
/home/s22463/yocto/sources/meta-openembedded/meta-webserver 6
# bitbake-layers show-appends
init-ifupdown_1.0.bb:
/home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifu
pdown_1.0.bbappend
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_%.bbappend
Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf
doesn't affect the order either.
If I rename
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_%.bbappend -> init-ifupdown_1.0.bbappend
the priority works, and my files in meta-platform are installed last, as I
want to overwrite all other files.
# mv
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_%.bbappend
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_1.0.bbappend
# bitbake-layers show-appends
init-ifupdown_1.0.bb:
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_1.0.bbappend
/home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifu
pdown_1.0.bbappend
The installation order works now, but I would rather avoid to patch the
other layers.
Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf
now, will affect the order!, and I can install meta-plaform either before or
after the meta-yougurt layer if I want.
Does anyone know the reason to the faulty sort-order?
/Christian Magnusson
On 11/24/2014 03:22 PM, Stevens, Nick wrote:
I think I've encountered a bug with how multiple bbappend files are
processed
when one of the bbappends contains a filename wildcard, but I want to make
sure
there's not something I'm missing before filing a bug report.
I have a BSP layer and a customization layer that are based on the OE/poky
Daisy release. The output of `bitbake-layers show-layers` looks like:
layer path priority
=====================================================
meta poky/meta 5
meta-yocto poky/meta-yocto 5
meta-yocto-bsp poky/meta-yocto-bsp 5
...ellided...
meta-bsp meta-bsp 6
meta-custom meta-custom 8
In meta-bsp there is a bbappend for base-files named
base-files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc
-
pretty straightforward:
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
SRC_URI += "file://sysctl.conf"
do_install_append() {
install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
}
Now what I want to do is add a file called base-files_%.bbappend to
meta-custom. It has its own version of sysctl.conf, and the recipe looks
like
this:
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
Here's where things get weird. If I run `bitbake -e base-files` and pull out
the section for FILESEXTRAPATHS, this is what I get (note that I've stripped
out the huge absolute paths to make this easier to read):
# $FILESEXTRAPATHS [5 operations]
# set poky/meta/conf/documentation.conf:172
# [doc] "Extends the search path the OpenEmbedded build system uses
when
looking for files and patches as it processes recipes and append files."
# _prepend
meta-custom/recipes-core/base-files/base-files_%.bbappend:6
# "meta-custom/recipes-core/base-files/base-files:"
# _prepend
meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
# "meta-bsp/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-custom/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
#
"meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-f
iles/base-files:"
# computed:
#
"meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-f
iles/base-files:"
FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-custom/rec
ipes-core/base-files/base-files:"
For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS,
even
though meta-custom has a higher priority!
I also have some questions about how FILESEXTRAPATHS is supposed to work
with append files and layer priorities.
Does a higher layer priority mean that the FILESEXTRAPATHS operation should
occur first? If this is the case, then a lower priority layer prepend will
appear to the left of the higher layer prepend since it runs later, and this
will probably not provide the result you want.
The prepend and layer logic seems messy to me. I would like to have a way to
write my custom layer and specify that what I include in my SRC_URI in each
recipe is definitive and can not be overwritten. Suggestions on how to best
accomplish this will be greatly appreciated.
I have been trying to accomplish this with FILESOVERRIDES, but this logic
seems counterintuitive since DISTROOVERRIDES take precedence over
MACHINEOVERRIDES in building the file search path during the unpack task.
It seems to me that the file search path should be built using
FILESOVERRIDES from left to right: TRANSLATED_TARGET_ARCH, MACHINEOVERRIDES,
and then DISTROOVERRIDES (specific to generic), but this isn't what I'm
seeing. Still investigating how it's all put together...
I verified this by building an image - the sysctl.conf that ends up in the
final image is the one from meta-bsp, not the one from meta-custom. But if I
switch the name of base-files_%.bbappend in meta-custom to
base-files_3.0.14.bbappend, this is what I get:
# $FILESEXTRAPATHS [5 operations]
# set poky/meta/conf/documentation.conf:172
# [doc] "Extends the search path the OpenEmbedded build system uses
when
looking for files and patches as it processes recipes and append files."
# _prepend
meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
# "meta-bsp/recipes-core/base-files/base-files:"
# _prepend
meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6
# "meta-custom/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-bsp/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
#
"meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-f
iles/base-files:"
# computed:
#
"meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-f
iles/base-files:"
FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta-bsp/rec
ipes-core/base-files/base-files:"
And now the sysctl.conf file is being pulled from meta-custom.
Is there something I'm missing, like some sort of decreased priorty for
bbappend files with a wildcard? Or is this a bug?
Thanks for any help!
Nick Stevens
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
<https://lists.yoctoproject.org/listinfo/yocto>
https://lists.yoctoproject.org/listinfo/yocto
[-- Attachment #2: Type: text/html, Size: 41539 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-05-16 18:42 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-24 20:22 Layer Priority with Wildcard .bbappend Files Stevens, Nick
2014-11-24 21:25 ` Bob Cochran
2014-11-25 11:12 ` Paul Eggleton
2014-11-25 18:04 ` Stevens, Nick
2014-11-25 20:38 ` Bob Cochran
2015-05-16 11:10 Christian Magnusson
2015-05-16 18:42 ` Christopher Larson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.