Problem add and removing files to filesystem

Problem add and removing files to filesystem

[Lorin Vandegrift]

[-- Attachment #1: Type: text/plain, Size: 715 bytes --]

Hi all,

We are having problems adding extra files/removing files from the generated
file system in Yocto.  We've had success with using a do_install command in
our own bb file and using commands in the format rsync -avz test.txt
${D}{sysconfdir}, however we can't write to some paths (i.e. writing to
root with ${D).  Additionally, we are not able to overwrite files generated
by other recipes.  We also tried using install and couldn't push our init
script (nstall -m 0755 test.sh ${D}${sysconfdir}/init.d).  Any advice for
these?  Also is there a recommended way for removing files?   We were able
to remove them by using the rm command and adding it to
ROOTFS_POSTPROCESS_COMMAND.

Thanks,
Lorin

[-- Attachment #2: Type: text/html, Size: 801 bytes --]

Re: Problem add and removing files to filesystem

[Khem Raj]

On 15-07-24 14:27:09, Lorin Vandegrift wrote:
> Hi all,
> 
> We are having problems adding extra files/removing files from the generated
> file system in Yocto.  We've had success with using a do_install command in
> our own bb file and using commands in the format rsync -avz test.txt
> ${D}{sysconfdir}, however we can't write to some paths (i.e. writing to
> root with ${D).  Additionally, we are not able to overwrite files generated
> by other recipes.  We also tried using install and couldn't push our init
> script (nstall -m 0755 test.sh ${D}${sysconfdir}/init.d).  Any advice for
> these?  Also is there a recommended way for removing files?   We were able
> to remove them by using the rm command and adding it to
> ROOTFS_POSTPROCESS_COMMAND.

You can consider few approaches

1. Move the files into packages of there own in the recipe which means

PACKAGES += "foo"
FILES_foo = "files you want to separate out"

2. Write a bbappend in your own layer if you are doing something thats
not generic and wont benefit community, in this bbappend you can write
do_install_append () function and do the removal and any other packaging
related adjustments needed for that recipe

3. Write your own rootfs post processing function and hook it up into
ROOTFS_POSTPROCESS_COMMAND variable, this will however mean that online
package upgrades wont work out of box.

If you are looking for overwriting files then consider them patching
using patches or overlaying files using SRC_URI itself again using a
bbappend, if its a config file and you have few tweaks you can sed/awk
then in do_install step itself.

what you are trying is violating the packaging concerns of buildsystem
and thats why its not working as you expected.

Consider components and files a component provides as basic unit of
operation, all changes has to be done at that level for best results.

-Khem

Re: Problem add and removing files to filesystem

[Lorin Vandegrift]

[-- Attachment #1: Type: text/plain, Size: 3380 bytes --]

On Fri, Jul 24, 2015 at 5:24 PM, Khem Raj <raj.khem@gmail.com> wrote:

> On 15-07-24 14:27:09, Lorin Vandegrift wrote:
> > Hi all,
> >
> > We are having problems adding extra files/removing files from the
> generated
> > file system in Yocto.  We've had success with using a do_install command
> in
> > our own bb file and using commands in the format rsync -avz test.txt
> > ${D}{sysconfdir}, however we can't write to some paths (i.e. writing to
> > root with ${D).  Additionally, we are not able to overwrite files
> generated
> > by other recipes.  We also tried using install and couldn't push our init
> > script (nstall -m 0755 test.sh ${D}${sysconfdir}/init.d).  Any advice for
> > these?  Also is there a recommended way for removing files?   We were
> able
> > to remove them by using the rm command and adding it to
> > ROOTFS_POSTPROCESS_COMMAND.
>
> You can consider few approaches
>
> 1. Move the files into packages of there own in the recipe which means
>
> PACKAGES += "foo"
> FILES_foo = "files you want to separate out"
>
> 2. Write a bbappend in your own layer if you are doing something thats
> not generic and wont benefit community, in this bbappend you can write
> do_install_append () function and do the removal and any other packaging
> related adjustments needed for that recipe
>
> 3. Write your own rootfs post processing function and hook it up into
> ROOTFS_POSTPROCESS_COMMAND variable, this will however mean that online
> package upgrades wont work out of box.
>
> If you are looking for overwriting files then consider them patching
> using patches or overlaying files using SRC_URI itself again using a
> bbappend, if its a config file and you have few tweaks you can sed/awk
> then in do_install step itself.
>
> what you are trying is violating the packaging concerns of buildsystem
> and thats why its not working as you expected.
>
> Consider components and files a component provides as basic unit of
> operation, all changes has to be done at that level for best results.
>
> -Khem
>


Hi there,

Thanks for the response. Unfortunately we tried your suggestions with no
luck. Below is the bb file for our package, do you see anything that we
might of missed? The tar ball listed below is in a subdirectory called
"files" in the same folder as the bb file.

Thanks

SUMMARY = "Extra filesystem files"
SECTION = "rootfs"
LICENSE = "CLOSED"

SRC_URI = "file://extrafiles.tar.gz"
S = "${WORKDIR}/files"

ROOTFS_POSTPROCESS_COMMAND += "set_root_passwd;"
ROOTFS_POSTPROCESS_COMMAND += "modify_filesystem;"

do_install() {
   install -m 0755 testFile.sh ${D}/home/root
   rsync -avz wpa_supplicant ${D}${sysconfdir}
   rsync -avz root ${D}${sysconfdir}
   rsync -avz network ${D}${sysconfdir}
   rsync -avz test.txt ${D}${sysconfdir}
   rsync -avz syslog.conf ${D}${sysconfdir}
   rsync -avz rules.d ${D}${sysconfdir}/udev/
   rsync -avz sys_init.service ${D}${sysconfdir}/system/
}

modify_filesystem() {
   rm ${IMAGE_ROOTFS}${sysconfdir}/wpa_supplicant.conf;
}

set_root_passwd() {
        sed 's%^root:[^:]*:%root:$1$xyz$KSp5j8kx3cPgA3/ODPZ6E.:%' \
        < ${IMAGE_ROOTFS}/etc/shadow \
        > ${IMAGE_ROOTFS}/etc/shadow.new;
        mv ${IMAGE_ROOTFS}/etc/shadow.new ${IMAGE_ROOTFS}/etc/shadow ;
}

EXTRA_IMAGE_FEATURES += "read-only-rootfs"

[-- Attachment #2: Type: text/html, Size: 4500 bytes --]

Problem add and removing files to filesystem

[Esponde, Joel]

[-- Attachment #1: Type: text/plain, Size: 2248 bytes --]

> On Mon Jul 27 10:53:36 PDT 2015, Lorin Vandegrift vandegriftlorin at gmail.com wrote:
>
> Hi there,
>
> Thanks for the response. Unfortunately we tried your suggestions with no
> luck. Below is the bb file for our package, do you see anything that we
> might of missed? The tar ball listed below is in a subdirectory called
> "files" in the same folder as the bb file.
>
> Thanks
>
> SUMMARY = "Extra filesystem files"
> SECTION = "rootfs"
> LICENSE = "CLOSED"
>
> SRC_URI = "file://extrafiles.tar.gz"
> S = "${WORKDIR}/files"
>
> ROOTFS_POSTPROCESS_COMMAND += "set_root_passwd;"
> ROOTFS_POSTPROCESS_COMMAND += "modify_filesystem;"
>
> do_install() {
>    install -m 0755 testFile.sh ${D}/home/root
>    rsync -avz wpa_supplicant ${D}${sysconfdir}
>    rsync -avz root ${D}${sysconfdir}
>    rsync -avz network ${D}${sysconfdir}
>    rsync -avz test.txt ${D}${sysconfdir}
>    rsync -avz syslog.conf ${D}${sysconfdir}
>    rsync -avz rules.d ${D}${sysconfdir}/udev/
>    rsync -avz sys_init.service ${D}${sysconfdir}/system/
> }
>
> modify_filesystem() {
>    rm ${IMAGE_ROOTFS}${sysconfdir}/wpa_supplicant.conf;
> }
>
> set_root_passwd() {
>         sed 's%^root:[^:]*:%root:$1$xyz$KSp5j8kx3cPgA3/ODPZ6E.:%' \
>         < ${IMAGE_ROOTFS}/etc/shadow \
>         > ${IMAGE_ROOTFS}/etc/shadow.new;
>        mv ${IMAGE_ROOTFS}/etc/shadow.new ${IMAGE_ROOTFS}/etc/shadow ;
> }
>
> EXTRA_IMAGE_FEATURES += "read-only-rootfs"

Hi,

I guess my answer will happen too late but it may be useful for other devs asking themselves the same question.
I faced the same issue today and at the end I managed to solve it by moving ROOTFS_POSTPROCESS_COMMAND directive and corresponding functions to the image's recipe.

There is a way to check if ROOTFS_POSTPROCESS_COMMAND is correctly set without having to build the image.
First call bitbake with the -e option and store the result in a file:
                bitbake core-image-minimal -e > core-image-minimal.env
Then edit core-image-minimal.env file, search for ROOTFS_POSTPROCESS_COMMAND variable and check that it has your function(s) in the list.
You can also check that the body of your functions are correctly set in the environment.

Joël




[-- Attachment #2: Type: text/html, Size: 12105 bytes --]