sysroot not being populated

sysroot not being populated

[Patrick Vacek]

Hello all,

I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr)
to provide an executable and a config file. The former recipe includes
`DEPENDS = "aktualizr-native"`, and my do_install() for
aktualizr-hsm-prov has a line something like this:

aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf

The binary executable (aktualizr) runs, which tells me that the recipe
can find that. (Although to be honest, I'm not sure which version it is
running.) However, it doesn't find the config file, and sure enough,
${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I
can see that aktualizr-native is populating its sysroot-destdir, but it
isn't getting copied to the sysroot for aktualizr-hsm-prov.

I see this problem in sumo and master, although previously this logic
has worked just fine in morty/pyro/rocko.

Switching to depending on aktualizr (instead of aktualizr-native) and
using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact
makes it worse, as it can't even find the aktualizr executable in that case.

What am I doing wrong? What changed between rocko and sumo?

Thanks,

Patrick Vacek
HERE Technologies

Re: sysroot not being populated

[Khem Raj]

Hi Patrick

On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
> Hello all,
>
> I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr)
> to provide an executable and a config file. The former recipe includes
> `DEPENDS = "aktualizr-native"`, and my do_install() for
> aktualizr-hsm-prov has a line something like this:
>
> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>
> The binary executable (aktualizr) runs, which tells me that the recipe
> can find that. (Although to be honest, I'm not sure which version it is
> running.) However, it doesn't find the config file, and sure enough,
> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I
> can see that aktualizr-native is populating its sysroot-destdir, but it
> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>
> I see this problem in sumo and master, although previously this logic
> has worked just fine in morty/pyro/rocko.
>
> Switching to depending on aktualizr (instead of aktualizr-native) and
> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact
> makes it worse, as it can't even find the aktualizr executable in that case.
>
> What am I doing wrong? What changed between rocko and sumo?

is aktualizr recipe installing this .conf file when building native
version. If not may be
add it to do_install

install -D -m 0644 <path/to/sota.conf> ${D}${libdir}/sota.conf

Re: sysroot not being populated

[Patrick Vacek]

On 07.06.2018 19:06, Khem Raj wrote:
> Hi Patrick
>
> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
>> Hello all,
>>
>> I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr)
>> to provide an executable and a config file. The former recipe includes
>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>> aktualizr-hsm-prov has a line something like this:
>>
>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>
>> The binary executable (aktualizr) runs, which tells me that the recipe
>> can find that. (Although to be honest, I'm not sure which version it is
>> running.) However, it doesn't find the config file, and sure enough,
>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I
>> can see that aktualizr-native is populating its sysroot-destdir, but it
>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>
>> I see this problem in sumo and master, although previously this logic
>> has worked just fine in morty/pyro/rocko.
>>
>> Switching to depending on aktualizr (instead of aktualizr-native) and
>> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact
>> makes it worse, as it can't even find the aktualizr executable in that case.
>>
>> What am I doing wrong? What changed between rocko and sumo?
> is aktualizr recipe installing this .conf file when building native
> version. If not may be
> add it to do_install
>
> install -D -m 0644 <path/to/sota.conf> ${D}${libdir}/sota.conf

Hello Khem,

I believe so. The aktualizr recipe includes the following (edited for
simplicity):

BBCLASSEXTEND =+ "native"

do_install_append () {
    install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf
}

PACKAGES =+ " ${PN}-host-tools "

FILES_${PN}-host-tools = " ${libdir}/sota.conf "

Thanks,
Patrick

Re: sysroot not being populated

[Khem Raj]

On 6/8/18 12:27 AM, Patrick Vacek wrote:
> On 07.06.2018 19:06, Khem Raj wrote:
>> Hi Patrick
>>
>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
>>> Hello all,
>>>
>>> I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr)
>>> to provide an executable and a config file. The former recipe includes
>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>> aktualizr-hsm-prov has a line something like this:
>>>
>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>
>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>> can find that. (Although to be honest, I'm not sure which version it is
>>> running.) However, it doesn't find the config file, and sure enough,
>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I
>>> can see that aktualizr-native is populating its sysroot-destdir, but it
>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>
>>> I see this problem in sumo and master, although previously this logic
>>> has worked just fine in morty/pyro/rocko.
>>>
>>> Switching to depending on aktualizr (instead of aktualizr-native) and
>>> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact
>>> makes it worse, as it can't even find the aktualizr executable in that case.
>>>
>>> What am I doing wrong? What changed between rocko and sumo?
>> is aktualizr recipe installing this .conf file when building native
>> version. If not may be
>> add it to do_install
>>
>> install -D -m 0644 <path/to/sota.conf> ${D}${libdir}/sota.conf
> 
> Hello Khem,
> 
> I believe so. The aktualizr recipe includes the following (edited for
> simplicity):
> 
> BBCLASSEXTEND =+ "native"
> 
> do_install_append () {
>      install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf
> }
> 
> PACKAGES =+ " ${PN}-host-tools "
> 
> FILES_${PN}-host-tools = " ${libdir}/sota.conf "
> 

Can you check location of sota.conf in the build tree for 
aktualizr-native in directory called package/

> Thanks,
> Patrick
> 

Re: sysroot not being populated

[Patrick Vacek]

On 08.06.2018 10:26, Khem Raj wrote:
>
>
> On 6/8/18 12:27 AM, Patrick Vacek wrote:
>> On 07.06.2018 19:06, Khem Raj wrote:
>>> Hi Patrick
>>>
>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
>>> <patrick.vacek@here.com> wrote:
>>>> Hello all,
>>>>
>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
>>>> (aktualizr)
>>>> to provide an executable and a config file. The former recipe includes
>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>>> aktualizr-hsm-prov has a line something like this:
>>>>
>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>>
>>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>>> can find that. (Although to be honest, I'm not sure which version
>>>> it is
>>>> running.) However, it doesn't find the config file, and sure enough,
>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
>>>> expect. I
>>>> can see that aktualizr-native is populating its sysroot-destdir,
>>>> but it
>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>>
>>>> I see this problem in sumo and master, although previously this logic
>>>> has worked just fine in morty/pyro/rocko.
>>>>
>>>> Switching to depending on aktualizr (instead of aktualizr-native) and
>>>> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in
>>>> fact
>>>> makes it worse, as it can't even find the aktualizr executable in
>>>> that case.
>>>>
>>>> What am I doing wrong? What changed between rocko and sumo?
>>> is aktualizr recipe installing this .conf file when building native
>>> version. If not may be
>>> add it to do_install
>>>
>>> install -D -m 0644 <path/to/sota.conf> ${D}${libdir}/sota.conf
>>
>> Hello Khem,
>>
>> I believe so. The aktualizr recipe includes the following (edited for
>> simplicity):
>>
>> BBCLASSEXTEND =+ "native"
>>
>> do_install_append () {
>>      install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf
>> }
>>
>> PACKAGES =+ " ${PN}-host-tools "
>>
>> FILES_${PN}-host-tools = " ${libdir}/sota.conf "
>>
>
> Can you check location of sota.conf in the build tree for
> aktualizr-native in directory called package/ 
Oddly, I do not see a package subdirectory inside the aktualizr-native
directory in the build tree. I do see it inside the aktualizr directory,
though, and it contains everything that I would expect. Is there some
sort of configuration of the packaging system for a native recipe that I
haven't done correctly?

For the record, here is the actual recipe:
https://github.com/advancedtelematic/meta-updater/blob/master/recipes-sota/aktualizr/aktualizr_git.bb

Re: sysroot not being populated

[Andre McCurdy]

On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
> On 08.06.2018 10:26, Khem Raj wrote:
>> On 6/8/18 12:27 AM, Patrick Vacek wrote:
>>> On 07.06.2018 19:06, Khem Raj wrote:
>>>> Hi Patrick
>>>>
>>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
>>>> <patrick.vacek@here.com> wrote:
>>>>> Hello all,
>>>>>
>>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
>>>>> (aktualizr)
>>>>> to provide an executable and a config file. The former recipe includes
>>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>>>> aktualizr-hsm-prov has a line something like this:
>>>>>
>>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>>>
>>>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>>>> can find that. (Although to be honest, I'm not sure which version
>>>>> it is
>>>>> running.) However, it doesn't find the config file, and sure enough,
>>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
>>>>> expect. I
>>>>> can see that aktualizr-native is populating its sysroot-destdir,
>>>>> but it
>>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>>>
>>>>> I see this problem in sumo and master, although previously this logic
>>>>> has worked just fine in morty/pyro/rocko.
>>>>>
>>>>> Switching to depending on aktualizr (instead of aktualizr-native) and
>>>>> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in
>>>>> fact
>>>>> makes it worse, as it can't even find the aktualizr executable in
>>>>> that case.
>>>>>
>>>>> What am I doing wrong? What changed between rocko and sumo?
>>>> is aktualizr recipe installing this .conf file when building native
>>>> version. If not may be
>>>> add it to do_install
>>>>
>>>> install -D -m 0644 <path/to/sota.conf> ${D}${libdir}/sota.conf
>>>
>>> Hello Khem,
>>>
>>> I believe so. The aktualizr recipe includes the following (edited for
>>> simplicity):
>>>
>>> BBCLASSEXTEND =+ "native"
>>>
>>> do_install_append () {
>>>      install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf
>>> }
>>>
>>> PACKAGES =+ " ${PN}-host-tools "
>>>
>>> FILES_${PN}-host-tools = " ${libdir}/sota.conf "
>>>
>>
>> Can you check location of sota.conf in the build tree for
>> aktualizr-native in directory called package/
> Oddly, I do not see a package subdirectory inside the aktualizr-native
> directory in the build tree. I do see it inside the aktualizr directory,
> though, and it contains everything that I would expect. Is there some
> sort of configuration of the packaging system for a native recipe that I
> haven't done correctly?

No, what you see is expected - there's no packaging for -native recipes.

Back to the original problem, I think you should consider the
aktualizr executable and the sota.conf file as two separate things.
The host executable should always be provided by aktualizr-native.
It's less clear what should provide sota.conf - it depends whether
it's just required for building other recipes (ie like a header file)
or whether it needs to be present in the final rootfs? If it's only
required for building other packages, then it should be in the -dev
package for aktualizr. If it needs to be present in the final rootfs
then it should be in the default package for aktualizr. Either way,
recipes which need to find sota.conf in sysroot would then depend on
"aktualizr". Therefore recipes which need both the host executable and
sota.conf in sysroot should depend on both aktualizr and
aktualizr-native.

> For the record, here is the actual recipe:
> https://github.com/advancedtelematic/meta-updater/blob/master/recipes-sota/aktualizr/aktualizr_git.bb

Re: sysroot not being populated

[Patrick Vacek]

On 12.06.2018 00:48, Andre McCurdy wrote:
> On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
>> On 08.06.2018 10:26, Khem Raj wrote:
>>> On 6/8/18 12:27 AM, Patrick Vacek wrote:
>>>> On 07.06.2018 19:06, Khem Raj wrote:
>>>>> Hi Patrick
>>>>>
>>>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
>>>>> <patrick.vacek@here.com> wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
>>>>>> (aktualizr)
>>>>>> to provide an executable and a config file. The former recipe includes
>>>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>>>>> aktualizr-hsm-prov has a line something like this:
>>>>>>
>>>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>>>>
>>>>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>>>>> can find that. (Although to be honest, I'm not sure which version
>>>>>> it is
>>>>>> running.) However, it doesn't find the config file, and sure enough,
>>>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
>>>>>> expect. I
>>>>>> can see that aktualizr-native is populating its sysroot-destdir,
>>>>>> but it
>>>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>>>>
>>>>>> I see this problem in sumo and master, although previously this logic
>>>>>> has worked just fine in morty/pyro/rocko.
>>>>>>
>>>>>> Switching to depending on aktualizr (instead of aktualizr-native) and
>>>>>> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in
>>>>>> fact
>>>>>> makes it worse, as it can't even find the aktualizr executable in
>>>>>> that case.
>>>>>>
>>>>>> What am I doing wrong? What changed between rocko and sumo?
>>>>> is aktualizr recipe installing this .conf file when building native
>>>>> version. If not may be
>>>>> add it to do_install
>>>>>
>>>>> install -D -m 0644 <path/to/sota.conf> ${D}${libdir}/sota.conf
>>>> Hello Khem,
>>>>
>>>> I believe so. The aktualizr recipe includes the following (edited for
>>>> simplicity):
>>>>
>>>> BBCLASSEXTEND =+ "native"
>>>>
>>>> do_install_append () {
>>>>      install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf
>>>> }
>>>>
>>>> PACKAGES =+ " ${PN}-host-tools "
>>>>
>>>> FILES_${PN}-host-tools = " ${libdir}/sota.conf "
>>>>
>>> Can you check location of sota.conf in the build tree for
>>> aktualizr-native in directory called package/
>> Oddly, I do not see a package subdirectory inside the aktualizr-native
>> directory in the build tree. I do see it inside the aktualizr directory,
>> though, and it contains everything that I would expect. Is there some
>> sort of configuration of the packaging system for a native recipe that I
>> haven't done correctly?
> No, what you see is expected - there's no packaging for -native recipes.
>
> Back to the original problem, I think you should consider the
> aktualizr executable and the sota.conf file as two separate things.
> The host executable should always be provided by aktualizr-native.
> It's less clear what should provide sota.conf - it depends whether
> it's just required for building other recipes (ie like a header file)
> or whether it needs to be present in the final rootfs? If it's only
> required for building other packages, then it should be in the -dev
> package for aktualizr. If it needs to be present in the final rootfs
> then it should be in the default package for aktualizr. Either way,
> recipes which need to find sota.conf in sysroot would then depend on
> "aktualizr". Therefore recipes which need both the host executable and
> sota.conf in sysroot should depend on both aktualizr and
> aktualizr-native.

Hello Andre,

Thanks for your insight. I didn't realize that there was no packaging
for native recipes (although that makes sense now that you've explained
it). In my use case, the second recipe just needs the config file from
the first for building a package, and it looks like your advice has
solved my problem perfectly. Thank you!

Patrick

Re: sysroot not being populated

[Patrick Vacek]

On 12.06.2018 00:48, Andre McCurdy wrote:
> On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
>> On 08.06.2018 10:26, Khem Raj wrote:
>>> On 6/8/18 12:27 AM, Patrick Vacek wrote:
>>>> On 07.06.2018 19:06, Khem Raj wrote:
>>>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
>>>>> <patrick.vacek@here.com> wrote:
>>>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
>>>>>> (aktualizr)
>>>>>> to provide an executable and a config file. The former recipe includes
>>>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>>>>> aktualizr-hsm-prov has a line something like this:
>>>>>>
>>>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>>>>
>>>>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>>>>> can find that. (Although to be honest, I'm not sure which version
>>>>>> it is
>>>>>> running.) However, it doesn't find the config file, and sure enough,
>>>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
>>>>>> expect. I
>>>>>> can see that aktualizr-native is populating its sysroot-destdir,
>>>>>> but it
>>>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>>>>
>>>>>> I see this problem in sumo and master, although previously this logic
>>>>>> has worked just fine in morty/pyro/rocko.
>>>>>>
>>>>>> What am I doing wrong? What changed between rocko and sumo?
>>> Can you check location of sota.conf in the build tree for
>>> aktualizr-native in directory called package/
>> Oddly, I do not see a package subdirectory inside the aktualizr-native
>> directory in the build tree. I do see it inside the aktualizr directory,
>> though, and it contains everything that I would expect. Is there some
>> sort of configuration of the packaging system for a native recipe that I
>> haven't done correctly?
> No, what you see is expected - there's no packaging for -native recipes.
>
> Back to the original problem, I think you should consider the
> aktualizr executable and the sota.conf file as two separate things.
> The host executable should always be provided by aktualizr-native.
> It's less clear what should provide sota.conf - it depends whether
> it's just required for building other recipes (ie like a header file)
> or whether it needs to be present in the final rootfs? If it's only
> required for building other packages, then it should be in the -dev
> package for aktualizr. If it needs to be present in the final rootfs
> then it should be in the default package for aktualizr. Either way,
> recipes which need to find sota.conf in sysroot would then depend on
> "aktualizr". Therefore recipes which need both the host executable and
> sota.conf in sysroot should depend on both aktualizr and
> aktualizr-native.
Actually, after some further research, it appears that my problem is not
entirely resolved. The explanation of packaging and native vs. target
dependencies all makes sense to me and usually works. However, this is
not the case for one specific condition which happens to be part of my
test suite.

One of my tests is to change the MACHINE in my local.conf, add some
additional layers, change some other variables, and then bitbake. Then I
undo that and bitbake again. For the most part, that works fine. In
rocko and before, it worked consistently. But it seems like on sumo and
master, when I do the test by changing the MACHINE and the layers, it
somehow clears out everything in my recipe's work directory, despite
that that recipe is not used by that test. In fact, it removes every
single file in tmp/work/core2-64-poky-linux (but leaves most of the
directories). When I switch back to how it was before and bitbake, most
of the files get recreated or re-copied, but not all. I have to
explicitly run `bitbake -c cleanall aktualizr aktualizr-hsm-prov` to get
things to work again. (Perhaps cleanall isn't necessary, but some form
of cleaning is, and it has to be both recipes.)

Can someone help explain why I am seeing this behavior? I don't
understand why the files get erased, why they don't get repopulated, and
why this was different back in rocko.

Thanks!
Patrick

Re: sysroot not being populated

[Khem Raj]

On Thu, Jun 14, 2018 at 5:45 AM Patrick Vacek <patrick.vacek@here.com> wrote:
>
> On 12.06.2018 00:48, Andre McCurdy wrote:
> > On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
> >> On 08.06.2018 10:26, Khem Raj wrote:
> >>> On 6/8/18 12:27 AM, Patrick Vacek wrote:
> >>>> On 07.06.2018 19:06, Khem Raj wrote:
> >>>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
> >>>>> <patrick.vacek@here.com> wrote:
> >>>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
> >>>>>> (aktualizr)
> >>>>>> to provide an executable and a config file. The former recipe includes
> >>>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
> >>>>>> aktualizr-hsm-prov has a line something like this:
> >>>>>>
> >>>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
> >>>>>>
> >>>>>> The binary executable (aktualizr) runs, which tells me that the recipe
> >>>>>> can find that. (Although to be honest, I'm not sure which version
> >>>>>> it is
> >>>>>> running.) However, it doesn't find the config file, and sure enough,
> >>>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
> >>>>>> expect. I
> >>>>>> can see that aktualizr-native is populating its sysroot-destdir,
> >>>>>> but it
> >>>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
> >>>>>>
> >>>>>> I see this problem in sumo and master, although previously this logic
> >>>>>> has worked just fine in morty/pyro/rocko.
> >>>>>>
> >>>>>> What am I doing wrong? What changed between rocko and sumo?
> >>> Can you check location of sota.conf in the build tree for
> >>> aktualizr-native in directory called package/
> >> Oddly, I do not see a package subdirectory inside the aktualizr-native
> >> directory in the build tree. I do see it inside the aktualizr directory,
> >> though, and it contains everything that I would expect. Is there some
> >> sort of configuration of the packaging system for a native recipe that I
> >> haven't done correctly?
> > No, what you see is expected - there's no packaging for -native recipes.
> >
> > Back to the original problem, I think you should consider the
> > aktualizr executable and the sota.conf file as two separate things.
> > The host executable should always be provided by aktualizr-native.
> > It's less clear what should provide sota.conf - it depends whether
> > it's just required for building other recipes (ie like a header file)
> > or whether it needs to be present in the final rootfs? If it's only
> > required for building other packages, then it should be in the -dev
> > package for aktualizr. If it needs to be present in the final rootfs
> > then it should be in the default package for aktualizr. Either way,
> > recipes which need to find sota.conf in sysroot would then depend on
> > "aktualizr". Therefore recipes which need both the host executable and
> > sota.conf in sysroot should depend on both aktualizr and
> > aktualizr-native.
> Actually, after some further research, it appears that my problem is not
> entirely resolved. The explanation of packaging and native vs. target
> dependencies all makes sense to me and usually works. However, this is
> not the case for one specific condition which happens to be part of my
> test suite.
>
> One of my tests is to change the MACHINE in my local.conf, add some
> additional layers, change some other variables, and then bitbake. Then I
> undo that and bitbake again. For the most part, that works fine. In
> rocko and before, it worked consistently. But it seems like on sumo and
> master, when I do the test by changing the MACHINE and the layers, it
> somehow clears out everything in my recipe's work directory, despite
> that that recipe is not used by that test. In fact, it removes every
> single file in tmp/work/core2-64-poky-linux (but leaves most of the
> directories). When I switch back to how it was before and bitbake, most
> of the files get recreated or re-copied, but not all. I have to
> explicitly run `bitbake -c cleanall aktualizr aktualizr-hsm-prov` to get
> things to work again. (Perhaps cleanall isn't necessary, but some form
> of cleaning is, and it has to be both recipes.)
>
> Can someone help explain why I am seeing this behavior? I don't
> understand why the files get erased, why they don't get repopulated, and
> why this was different back in rocko.
>

when machine and other variables are changed then its mostly
resulting in hash changes which is enforcing it to rebuild and when
rebuild happens it will clear the recipe specific build area for that
given recipe and repopulate it as it has to rebuild/repackage it since
this time it will get most of build artifacts from shared state as it
has been bullt with same hash once before. so it will most probably be
re-using that.

If doing clean build is helping then this might mean that they
are not staging all the files that you depend on. When you do
clean build they get staged into build area during compile phase and
it silently uses it. So ensure that all files you need are accounted
for in do_install

> Thanks!
> Patrick
>

Re: sysroot not being populated

[Patrick Vacek]

On 14.06.2018 16:28, Khem Raj wrote:
> On Thu, Jun 14, 2018 at 5:45 AM Patrick Vacek <patrick.vacek@here.com> wrote:
>> On 12.06.2018 00:48, Andre McCurdy wrote:
>>> On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek <patrick.vacek@here.com> wrote:
>>>> On 08.06.2018 10:26, Khem Raj wrote:
>>>>> On 6/8/18 12:27 AM, Patrick Vacek wrote:
>>>>>> On 07.06.2018 19:06, Khem Raj wrote:
>>>>>>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek
>>>>>>> <patrick.vacek@here.com> wrote:
>>>>>>>> I have a recipe (aktualizr-hsm-prov) that depends on another
>>>>>>>> (aktualizr)
>>>>>>>> to provide an executable and a config file. The former recipe includes
>>>>>>>> `DEPENDS = "aktualizr-native"`, and my do_install() for
>>>>>>>> aktualizr-hsm-prov has a line something like this:
>>>>>>>>
>>>>>>>> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf
>>>>>>>>
>>>>>>>> The binary executable (aktualizr) runs, which tells me that the recipe
>>>>>>>> can find that. (Although to be honest, I'm not sure which version
>>>>>>>> it is
>>>>>>>> running.) However, it doesn't find the config file, and sure enough,
>>>>>>>> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I
>>>>>>>> expect. I
>>>>>>>> can see that aktualizr-native is populating its sysroot-destdir,
>>>>>>>> but it
>>>>>>>> isn't getting copied to the sysroot for aktualizr-hsm-prov.
>>>>>>>>
>>>>>>>> I see this problem in sumo and master, although previously this logic
>>>>>>>> has worked just fine in morty/pyro/rocko.
>>>>>>>>
>>>>>>>> What am I doing wrong? What changed between rocko and sumo?
>>>>> Can you check location of sota.conf in the build tree for
>>>>> aktualizr-native in directory called package/
>>>> Oddly, I do not see a package subdirectory inside the aktualizr-native
>>>> directory in the build tree. I do see it inside the aktualizr directory,
>>>> though, and it contains everything that I would expect. Is there some
>>>> sort of configuration of the packaging system for a native recipe that I
>>>> haven't done correctly?
>>> No, what you see is expected - there's no packaging for -native recipes.
>>>
>>> Back to the original problem, I think you should consider the
>>> aktualizr executable and the sota.conf file as two separate things.
>>> The host executable should always be provided by aktualizr-native.
>>> It's less clear what should provide sota.conf - it depends whether
>>> it's just required for building other recipes (ie like a header file)
>>> or whether it needs to be present in the final rootfs? If it's only
>>> required for building other packages, then it should be in the -dev
>>> package for aktualizr. If it needs to be present in the final rootfs
>>> then it should be in the default package for aktualizr. Either way,
>>> recipes which need to find sota.conf in sysroot would then depend on
>>> "aktualizr". Therefore recipes which need both the host executable and
>>> sota.conf in sysroot should depend on both aktualizr and
>>> aktualizr-native.
>> Actually, after some further research, it appears that my problem is not
>> entirely resolved. The explanation of packaging and native vs. target
>> dependencies all makes sense to me and usually works. However, this is
>> not the case for one specific condition which happens to be part of my
>> test suite.
>>
>> One of my tests is to change the MACHINE in my local.conf, add some
>> additional layers, change some other variables, and then bitbake. Then I
>> undo that and bitbake again. For the most part, that works fine. In
>> rocko and before, it worked consistently. But it seems like on sumo and
>> master, when I do the test by changing the MACHINE and the layers, it
>> somehow clears out everything in my recipe's work directory, despite
>> that that recipe is not used by that test. In fact, it removes every
>> single file in tmp/work/core2-64-poky-linux (but leaves most of the
>> directories). When I switch back to how it was before and bitbake, most
>> of the files get recreated or re-copied, but not all. I have to
>> explicitly run `bitbake -c cleanall aktualizr aktualizr-hsm-prov` to get
>> things to work again. (Perhaps cleanall isn't necessary, but some form
>> of cleaning is, and it has to be both recipes.)
>>
>> Can someone help explain why I am seeing this behavior? I don't
>> understand why the files get erased, why they don't get repopulated, and
>> why this was different back in rocko.
>>
> when machine and other variables are changed then its mostly
> resulting in hash changes which is enforcing it to rebuild and when
> rebuild happens it will clear the recipe specific build area for that
> given recipe and repopulate it as it has to rebuild/repackage it since
> this time it will get most of build artifacts from shared state as it
> has been bullt with same hash once before. so it will most probably be
> re-using that.
That make sense.
> If doing clean build is helping then this might mean that they
> are not staging all the files that you depend on. When you do
> clean build they get staged into build area during compile phase and
> it silently uses it. So ensure that all files you need are accounted
> for in do_install
That makes sense, too, but the first recipe (aktualizr) definitely
installs the config file I need in do_install and lists it in
FILES_${PN}-host-tools. In fact, the entire package subdirectory of
aktualizr is not repopulated, and thus the recipe-sysroot for the second
recipe (aktualizr-hsm-prov, the one that depends on aktualizr) is also
left empty. It seems like something is not triggering or preventing the
repopulation of those directories when I change MACHINE back to the
original value (qemux86-64).