Installing a package on the board at runtime

Installing a package on the board at runtime

[Alan Martinovic]

Hey,
I know there is a method in devtool called devtool deploy-target.
It's a great thing to install a package on the board at runtime.

Is there something like that available without devtool?


A digression:
Am asking for a non devtool option because it
seems like devtool is ignoring the patch and config files
the recipe depends on (can't find any in workspace/sources).
My particular recipe has a custom build step that
depends on rendering config file templates (which fails due to missing
not finding the config file).

Re: Installing a package on the board at runtime

[Andrea Galbusera]

Hi Alan,

On Fri, Apr 27, 2018 at 1:38 PM, Alan Martinovic
<alan.martinovic@senic.com> wrote:
> Hey,
> I know there is a method in devtool called devtool deploy-target.
> It's a great thing to install a package on the board at runtime.
>
> Is there something like that available without devtool?

The devtool deploy-target way of deploying recipes' artefacts is
mostly oriented to easing development loops with the eSDK. A more
controlled way of installing packages would be to have
package-management added to your IMAGE_FEATURES [1] to leverage a
fully fledged package manager on the target (like any mainstream
distro). You can choose among different format by setting the
PACKAGE_CLASSES configuration variable. Using a package manager on the
target won't differ from how you install/upgrade your desktop
distribution. The only extra step you'll need to accomplish is setting
up a proper package feed to let your targets grab the packages from
your build/deploy/<package_format> dir. See the manual at [2] for more
details.

Also, devtool works at a recipe scope, not at package level. You can
imagine it as a sort of 'make install' that leverages ssh to work
update the target filesystem. Recipes can provide more than one
package, then only a proper package manager will let you select any of
the packages a recipe provides for installation/upgrade.

> A digression:
> Am asking for a non devtool option because it
> seems like devtool is ignoring the patch and config files
> the recipe depends on (can't find any in workspace/sources).

This shouldn't be the case. When using 'devtool modify'
workspace/sources/<your-recipe> gets populated with a git repository
which includes a 'devtool' branch which includes any patch the recipe
declare in its SRC_URI. Those patches are already applied to the
fetched sources in the form of git commits on the 'devtool' branch
[3].

> My particular recipe has a custom build step that
> depends on rendering config file templates (which fails due to missing
> not finding the config file).

This might be due to errors in setting up the extra task: difficult to
say without looking at how the custom step is defined. Maybe you can
share your recipe together with the error logs bitbake is reporting.

[1] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image
[2] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#runtime-package-management-server
[3] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#devtool-modifying-a-recipe

Re: Installing a package on the board at runtime

[Alan Martinovic]

Hey

> A more
controlled way of installing packages would be to have
package-management added to your IMAGE_FEATURES [1] to leverage a
fully fledged package manager on the target (like any mainstream
distro).

Yeah, I admit I have been ignorantly avoiding this, always thinking that
it isn't worth the overhead of setting the server and the additional
aps on the board.
Will check it out.

> This shouldn't be the case. When using 'devtool modify'
workspace/sources/<your-recipe> gets populated with a git repository
which includes a 'devtool' branch which includes any patch the recipe
declare in its SRC_URI.

Cool share about the devtool branch, tnx.

> Those patches are already applied to the
fetched sources in the form of git commits on the 'devtool' branch
[3].

I guess it isn't a patch that is directly applied to the source but a
config file that gets modified.

    ....
    SRC_URI += "file://bluetooth.conf"

    python do_render_templates() {
      render_template('bluetooth.conf')
    }

    addtask render_templates after do_compile before do_install

    do_install_append() {
        install -m 644 ${WORKDIR}/bluetooth.conf
${D}${sysconfdir}/dbus-1/system.d/bluetooth.conf
        install -d ${D}${systemd_unitdir}/system
        install -m 0644 ${WORKDIR}/bluetooth.service
${D}${systemd_unitdir}/system
    }
    do_render_templates[nostamp] = "1"


The render_template changes some entries in bluetooth.conf.
I guess when it's run from devshell it gets confused as where to look
for it.
Works ok with bitbake.

Be Well :)
Alan


On Fri, Apr 27, 2018 at 4:47 PM, Andrea Galbusera <gizero@gmail.com> wrote:
> Hi Alan,
>
> On Fri, Apr 27, 2018 at 1:38 PM, Alan Martinovic
> <alan.martinovic@senic.com> wrote:
>> Hey,
>> I know there is a method in devtool called devtool deploy-target.
>> It's a great thing to install a package on the board at runtime.
>>
>> Is there something like that available without devtool?
>
> The devtool deploy-target way of deploying recipes' artefacts is
> mostly oriented to easing development loops with the eSDK. A more
> controlled way of installing packages would be to have
> package-management added to your IMAGE_FEATURES [1] to leverage a
> fully fledged package manager on the target (like any mainstream
> distro). You can choose among different format by setting the
> PACKAGE_CLASSES configuration variable. Using a package manager on the
> target won't differ from how you install/upgrade your desktop
> distribution. The only extra step you'll need to accomplish is setting
> up a proper package feed to let your targets grab the packages from
> your build/deploy/<package_format> dir. See the manual at [2] for more
> details.
>
> Also, devtool works at a recipe scope, not at package level. You can
> imagine it as a sort of 'make install' that leverages ssh to work
> update the target filesystem. Recipes can provide more than one
> package, then only a proper package manager will let you select any of
> the packages a recipe provides for installation/upgrade.
>
>> A digression:
>> Am asking for a non devtool option because it
>> seems like devtool is ignoring the patch and config files
>> the recipe depends on (can't find any in workspace/sources).
>
> This shouldn't be the case. When using 'devtool modify'
> workspace/sources/<your-recipe> gets populated with a git repository
> which includes a 'devtool' branch which includes any patch the recipe
> declare in its SRC_URI. Those patches are already applied to the
> fetched sources in the form of git commits on the 'devtool' branch
> [3].
>
>> My particular recipe has a custom build step that
>> depends on rendering config file templates (which fails due to missing
>> not finding the config file).
>
> This might be due to errors in setting up the extra task: difficult to
> say without looking at how the custom step is defined. Maybe you can
> share your recipe together with the error logs bitbake is reporting.
>
> [1] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image
> [2] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#runtime-package-management-server
> [3] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#devtool-modifying-a-recipe

Re: Installing a package on the board at runtime

[Andrea Galbusera]

On Fri, Apr 27, 2018 at 5:39 PM, Alan Martinovic
<alan.martinovic@senic.com> wrote:
> Hey
>
>> A more
> controlled way of installing packages would be to have
> package-management added to your IMAGE_FEATURES [1] to leverage a
> fully fledged package manager on the target (like any mainstream
> distro).
>
> Yeah, I admit I have been ignorantly avoiding this, always thinking that
> it isn't worth the overhead of setting the server and the additional
> aps on the board.
> Will check it out.
>
>> This shouldn't be the case. When using 'devtool modify'
> workspace/sources/<your-recipe> gets populated with a git repository
> which includes a 'devtool' branch which includes any patch the recipe
> declare in its SRC_URI.
>
> Cool share about the devtool branch, tnx.
>
>> Those patches are already applied to the
> fetched sources in the form of git commits on the 'devtool' branch
> [3].
>
> I guess it isn't a patch that is directly applied to the source but a
> config file that gets modified.
>
>     ....
>     SRC_URI += "file://bluetooth.conf"
>
>     python do_render_templates() {
>       render_template('bluetooth.conf')
>     }

Can't see the render_template() implementation here, but my guess is
that, unless you are prepending any extra path to 'bluetooth.conf' you
are relying on the current working directory for the task, which
should be S.
While outside the devtool context S defaults to ${WORKDIR} (again,
unless you are setting it elsewhere), likely allowing
render_template() to find the file, when using devtool, S gets a
different value which comes from the externalsrc class and points to
the workspace.

My suggestion is trying to rework your render logic in a way that it
looks for source template in ${WORKDIR} and deploys the rendered
configuration file in ${D}, even better without overwriting the
template itself in ${WORKDIR}. You can find a similar pattern,
implemented as a shell snippet instead of a python function, i.e. in
meta/recipes-support/nspr/nspr_4.13.1.bb or
meta/recipes-support/nss/nss_3.28.1.bb from the poky repo, which do
both render their pkg-config templates that way and work seamlessly
either inside or outside devtool.

>     addtask render_templates after do_compile before do_install
>
>     do_install_append() {
>         install -m 644 ${WORKDIR}/bluetooth.conf
> ${D}${sysconfdir}/dbus-1/system.d/bluetooth.conf
>         install -d ${D}${systemd_unitdir}/system
>         install -m 0644 ${WORKDIR}/bluetooth.service
> ${D}${systemd_unitdir}/system
>     }
>     do_render_templates[nostamp] = "1"
>
>
> The render_template changes some entries in bluetooth.conf.
> I guess when it's run from devshell it gets confused as where to look
> for it.
> Works ok with bitbake.
>
> Be Well :)
> Alan
>
>
> On Fri, Apr 27, 2018 at 4:47 PM, Andrea Galbusera <gizero@gmail.com> wrote:
>> Hi Alan,
>>
>> On Fri, Apr 27, 2018 at 1:38 PM, Alan Martinovic
>> <alan.martinovic@senic.com> wrote:
>>> Hey,
>>> I know there is a method in devtool called devtool deploy-target.
>>> It's a great thing to install a package on the board at runtime.
>>>
>>> Is there something like that available without devtool?
>>
>> The devtool deploy-target way of deploying recipes' artefacts is
>> mostly oriented to easing development loops with the eSDK. A more
>> controlled way of installing packages would be to have
>> package-management added to your IMAGE_FEATURES [1] to leverage a
>> fully fledged package manager on the target (like any mainstream
>> distro). You can choose among different format by setting the
>> PACKAGE_CLASSES configuration variable. Using a package manager on the
>> target won't differ from how you install/upgrade your desktop
>> distribution. The only extra step you'll need to accomplish is setting
>> up a proper package feed to let your targets grab the packages from
>> your build/deploy/<package_format> dir. See the manual at [2] for more
>> details.
>>
>> Also, devtool works at a recipe scope, not at package level. You can
>> imagine it as a sort of 'make install' that leverages ssh to work
>> update the target filesystem. Recipes can provide more than one
>> package, then only a proper package manager will let you select any of
>> the packages a recipe provides for installation/upgrade.
>>
>>> A digression:
>>> Am asking for a non devtool option because it
>>> seems like devtool is ignoring the patch and config files
>>> the recipe depends on (can't find any in workspace/sources).
>>
>> This shouldn't be the case. When using 'devtool modify'
>> workspace/sources/<your-recipe> gets populated with a git repository
>> which includes a 'devtool' branch which includes any patch the recipe
>> declare in its SRC_URI. Those patches are already applied to the
>> fetched sources in the form of git commits on the 'devtool' branch
>> [3].
>>
>>> My particular recipe has a custom build step that
>>> depends on rendering config file templates (which fails due to missing
>>> not finding the config file).
>>
>> This might be due to errors in setting up the extra task: difficult to
>> say without looking at how the custom step is defined. Maybe you can
>> share your recipe together with the error logs bitbake is reporting.
>>
>> [1] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#ref-features-image
>> [2] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#runtime-package-management-server
>> [3] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#devtool-modifying-a-recipe