Files missing from package

Files missing from package

[Gary Thomas]

I'm building a package(recipe) for a system which uses opkg
as the packaging tool.  In my recipe, I have these additional
lines (only the relevant parts shown):

   do_install_append() {
       install -d ${D}${sbindir}
       install -d ${D}/etc
       install -m 0755 ${S}/tools/get_tool ${D}${sbindir}
       ln -s get_tool ${D}${sbindir}/put_tool
       install -m 0644 ${WORKDIR}/tool.config ${D}/etc
   }

   PACKAGES += "${PN}-tools"
   FILES_${PN}-tools = "${sbindir}/get_tool \
   		     ${sbindir}/put_tool \
                        /etc/tool.config \
   "

When the recipe is built, I get the correct files in my image:
   $ find tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/etc
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/etc/tool.config
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin/get_tool
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin/put_tool
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/bin
   tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/bin/some_tool

However, the 'some-tool-tools' package it missing the '/etc/tools.config' file.
I unpacked it manually to see this:

   $ tar -ztvf data.tar.gz
   drwxr-xr-x root/root         0 2012-10-08 09:28 ./
   drwxr-xr-x root/root         0 2012-10-08 09:28 ./usr/
   drwxr-xr-x root/root         0 2012-10-08 09:28 ./usr/sbin/
   lrwxrwxrwx root/root         0 2012-10-08 09:28 ./usr/sbin/put_tool -> get_tool
   -rwxr-xr-x root/root     15044 2012-10-08 09:28 ./usr/sbin/get_tool

There is no '/etc' directory to be seen.

What am I doing wrong?  This is built using poky:master '7c39c87d52c20e47cf90275a16e4517a296c8388'
from just a few days ago.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Files missing from package

[Paul Eggleton]

On Monday 08 October 2012 10:13:44 Gary Thomas wrote:
> I'm building a package(recipe) for a system which uses opkg
> as the packaging tool.  In my recipe, I have these additional
> lines (only the relevant parts shown):
> 
>    do_install_append() {
>        install -d ${D}${sbindir}
>        install -d ${D}/etc
>        install -m 0755 ${S}/tools/get_tool ${D}${sbindir}
>        ln -s get_tool ${D}${sbindir}/put_tool
>        install -m 0644 ${WORKDIR}/tool.config ${D}/etc
>    }
> 
>    PACKAGES += "${PN}-tools"
>    FILES_${PN}-tools = "${sbindir}/get_tool \
>    		     ${sbindir}/put_tool \
>                         /etc/tool.config \
>    "
> 
> When the recipe is built, I get the correct files in my image:
>    $ find tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/
>    tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/
>    tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/etc
>   
> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/etc/tool.conf
> ig tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr
> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin
> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin/get_
> tool
> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin/put_
> tool tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/bin
> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/bin/some_tool
> 
> However, the 'some-tool-tools' package it missing the '/etc/tools.config'
> file. I unpacked it manually to see this:
> 
>    $ tar -ztvf data.tar.gz
>    drwxr-xr-x root/root         0 2012-10-08 09:28 ./
>    drwxr-xr-x root/root         0 2012-10-08 09:28 ./usr/
>    drwxr-xr-x root/root         0 2012-10-08 09:28 ./usr/sbin/
>    lrwxrwxrwx root/root         0 2012-10-08 09:28 ./usr/sbin/put_tool ->
> get_tool -rwxr-xr-x root/root     15044 2012-10-08 09:28
> ./usr/sbin/get_tool
> 
> There is no '/etc' directory to be seen.
> 
> What am I doing wrong?  This is built using poky:master
> '7c39c87d52c20e47cf90275a16e4517a296c8388' from just a few days ago.

Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default 
value of FILES_${PN}, and your additional package is appended to PACKAGES, I 
suspect you'll find the file is being picked up by the main package first. You 
need to use =+ instead of += so that the some-tool-tools package is prepended 
to PACKAGES and thus gets the file before the main package.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Files missing from package

[Gary Thomas]

On 2012-10-08 10:24, Paul Eggleton wrote:
> On Monday 08 October 2012 10:13:44 Gary Thomas wrote:
>> I'm building a package(recipe) for a system which uses opkg
>> as the packaging tool.  In my recipe, I have these additional
>> lines (only the relevant parts shown):
>>
>>     do_install_append() {
>>         install -d ${D}${sbindir}
>>         install -d ${D}/etc
>>         install -m 0755 ${S}/tools/get_tool ${D}${sbindir}
>>         ln -s get_tool ${D}${sbindir}/put_tool
>>         install -m 0644 ${WORKDIR}/tool.config ${D}/etc
>>     }
>>
>>     PACKAGES += "${PN}-tools"
>>     FILES_${PN}-tools = "${sbindir}/get_tool \
>>     		     ${sbindir}/put_tool \
>>                          /etc/tool.config \
>>     "
>>
>> When the recipe is built, I get the correct files in my image:
>>     $ find tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/
>>     tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/
>>     tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/etc
>>
>> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/etc/tool.conf
>> ig tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr
>> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin
>> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin/get_
>> tool
>> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/usr/sbin/put_
>> tool tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/bin
>> tmp/work/mytarget-poky-linux-gnueabi/some-tool-1.0-r0.1/image/bin/some_tool
>>
>> However, the 'some-tool-tools' package it missing the '/etc/tools.config'
>> file. I unpacked it manually to see this:
>>
>>     $ tar -ztvf data.tar.gz
>>     drwxr-xr-x root/root         0 2012-10-08 09:28 ./
>>     drwxr-xr-x root/root         0 2012-10-08 09:28 ./usr/
>>     drwxr-xr-x root/root         0 2012-10-08 09:28 ./usr/sbin/
>>     lrwxrwxrwx root/root         0 2012-10-08 09:28 ./usr/sbin/put_tool ->
>> get_tool -rwxr-xr-x root/root     15044 2012-10-08 09:28
>> ./usr/sbin/get_tool
>>
>> There is no '/etc' directory to be seen.
>>
>> What am I doing wrong?  This is built using poky:master
>> '7c39c87d52c20e47cf90275a16e4517a296c8388' from just a few days ago.
>
> Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default
> value of FILES_${PN}, and your additional package is appended to PACKAGES, I
> suspect you'll find the file is being picked up by the main package first. You
> need to use =+ instead of += so that the some-tool-tools package is prepended
> to PACKAGES and thus gets the file before the main package.

Pretty obscure, but that did indeed solve the problem.

Thanks!

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Files missing from package

[Tomas Frydrych]

On 08/10/12 17:24, Paul Eggleton wrote:
> Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default 
> value of FILES_${PN}, and your additional package is appended to PACKAGES, I 
> suspect you'll find the file is being picked up by the main package first. You 
> need to use =+ instead of += so that the some-tool-tools package is prepended 
> to PACKAGES and thus gets the file before the main package.

Probably worth noting that using =+ with PACKAGES can have some
undesirable side effects. For example, if the recipe in question
generates packages dynamically, e.g., for plugins, the first package in
the PACKAGES list will be considered the 'main' package, and the dynamic
packages will RDEPEND on it. If you modify PACKAGES in such a recipe
using the =+, this will result in broken dependency chain without any
obvious signs something is not right. I think it's much better to
explicitly specify what should go into the main package than using =+ to
work around the unsuitable main package default.

Tomas

Re: Files missing from package

[Paul Eggleton]

On Monday 08 October 2012 18:40:45 Tomas Frydrych wrote:
> On 08/10/12 17:24, Paul Eggleton wrote:
> > Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default
> > value of FILES_${PN}, and your additional package is appended to PACKAGES,
> > I suspect you'll find the file is being picked up by the main package
> > first. You need to use =+ instead of += so that the some-tool-tools
> > package is prepended to PACKAGES and thus gets the file before the main
> > package.
> 
> Probably worth noting that using =+ with PACKAGES can have some
> undesirable side effects. For example, if the recipe in question
> generates packages dynamically, e.g., for plugins, the first package in
> the PACKAGES list will be considered the 'main' package, and the dynamic
> packages will RDEPEND on it. If you modify PACKAGES in such a recipe
> using the =+, this will result in broken dependency chain without any
> obvious signs something is not right. I think it's much better to
> explicitly specify what should go into the main package than using =+ to
> work around the unsuitable main package default.

If we're using just the first item in PACKAGES for anything anywhere then that 
is wrong and we should fix it. I would be surprised if we were still doing that 
(other than the old fallback code in update-rc.d.bbclass that is probably 
never hit) since the first item in PACKAGES hasn't been ${PN} for some time now 
- in fact it's currently the last item if you look at the default value of 
PACKAGES in conf/bitbake.conf in master.

In master and the upcoming release we do have PACKAGE_BEFORE_PN, which may be 
more appropriate for adding packages such as this one.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Files missing from package

[Tomas Frydrych]

On 08/10/12 19:22, Paul Eggleton wrote:
> On Monday 08 October 2012 18:40:45 Tomas Frydrych wrote:
>> On 08/10/12 17:24, Paul Eggleton wrote:
>>> Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default
>>> value of FILES_${PN}, and your additional package is appended to PACKAGES,
>>> I suspect you'll find the file is being picked up by the main package
>>> first. You need to use =+ instead of += so that the some-tool-tools
>>> package is prepended to PACKAGES and thus gets the file before the main
>>> package.
>>
>> Probably worth noting that using =+ with PACKAGES can have some
>> undesirable side effects. For example, if the recipe in question
>> generates packages dynamically, e.g., for plugins, the first package in
>> the PACKAGES list will be considered the 'main' package, and the dynamic
>> packages will RDEPEND on it. If you modify PACKAGES in such a recipe
>> using the =+, this will result in broken dependency chain without any
>> obvious signs something is not right. I think it's much better to
>> explicitly specify what should go into the main package than using =+ to
>> work around the unsuitable main package default.
> 
> If we're using just the first item in PACKAGES for anything anywhere then that 
> is wrong and we should fix it. I would be surprised if we were still doing that 

Denzil does,

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/classes/package.bbclass?h=denzil#n110

Tomas

Re: Files missing from package

[Richard Purdie]

On Mon, 2012-10-08 at 18:40 +0100, Tomas Frydrych wrote:
> On 08/10/12 17:24, Paul Eggleton wrote:
> > Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default 
> > value of FILES_${PN}, and your additional package is appended to PACKAGES, I 
> > suspect you'll find the file is being picked up by the main package first. You 
> > need to use =+ instead of += so that the some-tool-tools package is prepended 
> > to PACKAGES and thus gets the file before the main package.
> 
> Probably worth noting that using =+ with PACKAGES can have some
> undesirable side effects. For example, if the recipe in question
> generates packages dynamically, e.g., for plugins, the first package in
> the PACKAGES list will be considered the 'main' package, and the dynamic
> packages will RDEPEND on it. If you modify PACKAGES in such a recipe
> using the =+, this will result in broken dependency chain without any
> obvious signs something is not right. I think it's much better to
> explicitly specify what should go into the main package than using =+ to
> work around the unsuitable main package default.

FWIW, there is the PACKAGE_BEFORE_PN variable for this and other
scenarios where you want something just before PN in PACKAGES.

Cheers,

Richard

Re: Files missing from package

[Gary Thomas]

On 2012-10-08 14:37, Richard Purdie wrote:
> On Mon, 2012-10-08 at 18:40 +0100, Tomas Frydrych wrote:
>> On 08/10/12 17:24, Paul Eggleton wrote:
>>> Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default
>>> value of FILES_${PN}, and your additional package is appended to PACKAGES, I
>>> suspect you'll find the file is being picked up by the main package first. You
>>> need to use =+ instead of += so that the some-tool-tools package is prepended
>>> to PACKAGES and thus gets the file before the main package.
>>
>> Probably worth noting that using =+ with PACKAGES can have some
>> undesirable side effects. For example, if the recipe in question
>> generates packages dynamically, e.g., for plugins, the first package in
>> the PACKAGES list will be considered the 'main' package, and the dynamic
>> packages will RDEPEND on it. If you modify PACKAGES in such a recipe
>> using the =+, this will result in broken dependency chain without any
>> obvious signs something is not right. I think it's much better to
>> explicitly specify what should go into the main package than using =+ to
>> work around the unsuitable main package default.
>
> FWIW, there is the PACKAGE_BEFORE_PN variable for this and other
> scenarios where you want something just before PN in PACKAGES.

This does sound better, I always like to be explicit, especially when
the default/background has a possibility of biting me!

So, how would I use that for my case (i.e. what lines do I add/change
in my recipe to use it)?

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Files missing from package

[Richard Purdie]

On Mon, 2012-10-08 at 16:22 -0600, Gary Thomas wrote:
> On 2012-10-08 14:37, Richard Purdie wrote:
> > On Mon, 2012-10-08 at 18:40 +0100, Tomas Frydrych wrote:
> >> On 08/10/12 17:24, Paul Eggleton wrote:
> >>> Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default
> >>> value of FILES_${PN}, and your additional package is appended to PACKAGES, I
> >>> suspect you'll find the file is being picked up by the main package first. You
> >>> need to use =+ instead of += so that the some-tool-tools package is prepended
> >>> to PACKAGES and thus gets the file before the main package.
> >>
> >> Probably worth noting that using =+ with PACKAGES can have some
> >> undesirable side effects. For example, if the recipe in question
> >> generates packages dynamically, e.g., for plugins, the first package in
> >> the PACKAGES list will be considered the 'main' package, and the dynamic
> >> packages will RDEPEND on it. If you modify PACKAGES in such a recipe
> >> using the =+, this will result in broken dependency chain without any
> >> obvious signs something is not right. I think it's much better to
> >> explicitly specify what should go into the main package than using =+ to
> >> work around the unsuitable main package default.
> >
> > FWIW, there is the PACKAGE_BEFORE_PN variable for this and other
> > scenarios where you want something just before PN in PACKAGES.
> 
> This does sound better, I always like to be explicit, especially when
> the default/background has a possibility of biting me!
> 
> So, how would I use that for my case (i.e. what lines do I add/change
> in my recipe to use it)?

PACKAGE_BEFORE_PN += "${PN}-tools"
FILES_${PN}-tools = "${sbindir}/get_tool \
                     ${sbindir}/put_tool \
                        /etc/tool.config \


PACKAGE_BEFORE_PN lets you put the package specifically after
-dev/-dbg/-doc/-locale and so on but before PN.

Cheers,

Richard

Re: Files missing from package

[Gary Thomas]

On 2012-10-08 16:39, Richard Purdie wrote:
> On Mon, 2012-10-08 at 16:22 -0600, Gary Thomas wrote:
>> On 2012-10-08 14:37, Richard Purdie wrote:
>>> On Mon, 2012-10-08 at 18:40 +0100, Tomas Frydrych wrote:
>>>> On 08/10/12 17:24, Paul Eggleton wrote:
>>>>> Since ${sysconfdir} is /etc by default and ${sysconfdir} is in the default
>>>>> value of FILES_${PN}, and your additional package is appended to PACKAGES, I
>>>>> suspect you'll find the file is being picked up by the main package first. You
>>>>> need to use =+ instead of += so that the some-tool-tools package is prepended
>>>>> to PACKAGES and thus gets the file before the main package.
>>>>
>>>> Probably worth noting that using =+ with PACKAGES can have some
>>>> undesirable side effects. For example, if the recipe in question
>>>> generates packages dynamically, e.g., for plugins, the first package in
>>>> the PACKAGES list will be considered the 'main' package, and the dynamic
>>>> packages will RDEPEND on it. If you modify PACKAGES in such a recipe
>>>> using the =+, this will result in broken dependency chain without any
>>>> obvious signs something is not right. I think it's much better to
>>>> explicitly specify what should go into the main package than using =+ to
>>>> work around the unsuitable main package default.
>>>
>>> FWIW, there is the PACKAGE_BEFORE_PN variable for this and other
>>> scenarios where you want something just before PN in PACKAGES.
>>
>> This does sound better, I always like to be explicit, especially when
>> the default/background has a possibility of biting me!
>>
>> So, how would I use that for my case (i.e. what lines do I add/change
>> in my recipe to use it)?
>
> PACKAGE_BEFORE_PN += "${PN}-tools"
> FILES_${PN}-tools = "${sbindir}/get_tool \
>                       ${sbindir}/put_tool \
>                          /etc/tool.config \
>
>
> PACKAGE_BEFORE_PN lets you put the package specifically after
> -dev/-dbg/-doc/-locale and so on but before PN.

I needed to write this as
   PACKAGES_BEFORE_${PN} += "${PN}-tools"

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Files missing from package

[Paul Eggleton]

On Tuesday 09 October 2012 06:36:23 Gary Thomas wrote:
> On 2012-10-08 16:39, Richard Purdie wrote:
> > PACKAGE_BEFORE_PN lets you put the package specifically after
> > -dev/-dbg/-doc/-locale and so on but before PN.
> 
> I needed to write this as
>    PACKAGES_BEFORE_${PN} += "${PN}-tools"

Just to be clear the variable is called PACKAGE_BEFORE_PN and not 
PACKAGES_BEFORE_PN; otherwise yes.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Files missing from package

[Gary Thomas]

On 2012-10-09 06:41, Paul Eggleton wrote:
> On Tuesday 09 October 2012 06:36:23 Gary Thomas wrote:
>> On 2012-10-08 16:39, Richard Purdie wrote:
>>> PACKAGE_BEFORE_PN lets you put the package specifically after
>>> -dev/-dbg/-doc/-locale and so on but before PN.
>>
>> I needed to write this as
>>     PACKAGES_BEFORE_${PN} += "${PN}-tools"
>
> Just to be clear the variable is called PACKAGE_BEFORE_PN and not
> PACKAGES_BEFORE_PN; otherwise yes.

Indeed, that does work.  Sorry about my typo...

How did 'PACKAGES_BEFORE_${PN}' manage to work?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Files missing from package

[Paul Eggleton]

On Tuesday 09 October 2012 06:46:02 Gary Thomas wrote:
> On 2012-10-09 06:41, Paul Eggleton wrote:
> > On Tuesday 09 October 2012 06:36:23 Gary Thomas wrote:
> >> On 2012-10-08 16:39, Richard Purdie wrote:
> >>> PACKAGE_BEFORE_PN lets you put the package specifically after
> >>> -dev/-dbg/-doc/-locale and so on but before PN.
> >> 
> >> I needed to write this as
> >> 
> >>     PACKAGES_BEFORE_${PN} += "${PN}-tools"
> > 
> > Just to be clear the variable is called PACKAGE_BEFORE_PN and not
> > PACKAGES_BEFORE_PN; otherwise yes.
> 
> Indeed, that does work.  Sorry about my typo...
> 
> How did 'PACKAGES_BEFORE_${PN}' manage to work?

I'm not sure how it could have unless you have some additional statement or 
code that is getting the package into PACKAGES. By all means, change it back 
and see if your package appears in the final value of PACKAGES using:

bitbake -e recipename | grep ^PACKAGES=

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Files missing from package

[Gary Thomas]

On 2012-10-09 06:48, Paul Eggleton wrote:
> On Tuesday 09 October 2012 06:46:02 Gary Thomas wrote:
>> On 2012-10-09 06:41, Paul Eggleton wrote:
>>> On Tuesday 09 October 2012 06:36:23 Gary Thomas wrote:
>>>> On 2012-10-08 16:39, Richard Purdie wrote:
>>>>> PACKAGE_BEFORE_PN lets you put the package specifically after
>>>>> -dev/-dbg/-doc/-locale and so on but before PN.
>>>>
>>>> I needed to write this as
>>>>
>>>>      PACKAGES_BEFORE_${PN} += "${PN}-tools"
>>>
>>> Just to be clear the variable is called PACKAGE_BEFORE_PN and not
>>> PACKAGES_BEFORE_PN; otherwise yes.
>>
>> Indeed, that does work.  Sorry about my typo...
>>
>> How did 'PACKAGES_BEFORE_${PN}' manage to work?
>
> I'm not sure how it could have unless you have some additional statement or
> code that is getting the package into PACKAGES. By all means, change it back
> and see if your package appears in the final value of PACKAGES using:
>
> bitbake -e recipename | grep ^PACKAGES=

Right again :-)  What I thought was working was just a left over .ipk file.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------