eSDK install script failure

eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 1696 bytes --]

Seeing the errors below while installing an eSDK. This is a routinely
generated VM that installs the eSDK from installation script. The errors
appeared with the latest iteration of the eSDK script, which is generated
with almost up-to-date revisions from master. Of course I have extra layers
in the mix, but none of them apparently had relevant changed since last
(working) iteration: mostly synching to master branches happened. Can
anyone help suggesting how to investigate this further? What do those
unexpected task mean? I'm blocked on releasing this SDK to developers and
clues from expert would be very appreciated...

==> default: Checking sstate mirror object availability...
==> default: done.
==> default: ERROR: Task python-native.do_fetch attempted to execute
unexpectedly
==> default: ERROR: Task python-native.do_prepare_recipe_sysroot attempted
to execute unexpectedly
==> default: ERROR: Task python-native.do_unpack attempted to execute
unexpectedly
==> default: ERROR: Task python-native.do_patch attempted to execute
unexpectedly
==> default: ERROR: Task python-native.do_populate_lic attempted to execute
unexpectedly and should have been setscened
==> default: ERROR: Task python-native.do_configure attempted to execute
unexpectedly
==> default: ERROR: Task python-native.do_compile attempted to execute
unexpectedly
==> default: ERROR: Task python-native.do_install attempted to execute
unexpectedly
==> default: ERROR: Task python-native.do_populate_sysroot attempted to
execute unexpectedly and should have been setscened
==> default: ERROR: SDK preparation failed: error log written to
/home/vagrant/poky_sdk/preparing_build_system.log

[-- Attachment #2: Type: text/html, Size: 1773 bytes --]

Re: eSDK install script failure

[Paul Eggleton]

Hi Andrea,

On Wednesday, 20 September 2017 8:44:22 PM NZST Andrea Galbusera wrote:
> Seeing the errors below while installing an eSDK. This is a routinely
> generated VM that installs the eSDK from installation script. The errors
> appeared with the latest iteration of the eSDK script, which is generated
> with almost up-to-date revisions from master. Of course I have extra layers
> in the mix, but none of them apparently had relevant changed since last
> (working) iteration: mostly synching to master branches happened. Can
> anyone help suggesting how to investigate this further? What do those
> unexpected task mean? I'm blocked on releasing this SDK to developers and
> clues from expert would be very appreciated...
> 
> ==> default: Checking sstate mirror object availability...
> ==> default: done.
> ==> default: ERROR: Task python-native.do_fetch attempted to execute
> unexpectedly
> ==> default: ERROR: Task python-native.do_prepare_recipe_sysroot attempted
> to execute unexpectedly
> ==> default: ERROR: Task python-native.do_unpack attempted to execute
> unexpectedly
> ==> default: ERROR: Task python-native.do_patch attempted to execute
> unexpectedly
> ==> default: ERROR: Task python-native.do_populate_lic attempted to execute
> unexpectedly and should have been setscened
> ==> default: ERROR: Task python-native.do_configure attempted to execute
> unexpectedly
> ==> default: ERROR: Task python-native.do_compile attempted to execute
> unexpectedly
> ==> default: ERROR: Task python-native.do_install attempted to execute
> unexpectedly
> ==> default: ERROR: Task python-native.do_populate_sysroot attempted to
> execute unexpectedly and should have been setscened
> ==> default: ERROR: SDK preparation failed: error log written to
> /home/vagrant/poky_sdk/preparing_build_system.log
> 

Basically this means that these tasks tried to execute where really the 
results should have been able to be restored from sstate. 

The cause of this type of error is one of three things: 

1) The sstate archive corresponding to a task wasn't able to be fetched from 
the server (for a minimal eSDK) or wasn't present in the installer (for a full 
eSDK - less likely as we basically do a trial run as part of building the eSDK 
in the first place)

2) The signature was somehow different to what it should have been. (Locked 
signatures are supposed to guard against this.)

3) A task that wasn't expected to execute did execute and thus the sstate 
wasn't available.

Given that this was python-native which I would expect would be a normal part 
of the SDK, I would suspect #1. Is this a minimal or full eSDK (i.e. what is 
SDK_EXT_TYPE set to)?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 3342 bytes --]

Hi Paul,
thanks for explaining and helping sorting this out.

On Wed, Sep 20, 2017 at 11:54 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> Hi Andrea,
>
> On Wednesday, 20 September 2017 8:44:22 PM NZST Andrea Galbusera wrote:
> > Seeing the errors below while installing an eSDK. This is a routinely
> > generated VM that installs the eSDK from installation script. The errors
> > appeared with the latest iteration of the eSDK script, which is generated
> > with almost up-to-date revisions from master. Of course I have extra
> layers
> > in the mix, but none of them apparently had relevant changed since last
> > (working) iteration: mostly synching to master branches happened. Can
> > anyone help suggesting how to investigate this further? What do those
> > unexpected task mean? I'm blocked on releasing this SDK to developers and
> > clues from expert would be very appreciated...
> >
> > ==> default: Checking sstate mirror object availability...
> > ==> default: done.
> > ==> default: ERROR: Task python-native.do_fetch attempted to execute
> > unexpectedly
> > ==> default: ERROR: Task python-native.do_prepare_recipe_sysroot
> attempted
> > to execute unexpectedly
> > ==> default: ERROR: Task python-native.do_unpack attempted to execute
> > unexpectedly
> > ==> default: ERROR: Task python-native.do_patch attempted to execute
> > unexpectedly
> > ==> default: ERROR: Task python-native.do_populate_lic attempted to
> execute
> > unexpectedly and should have been setscened
> > ==> default: ERROR: Task python-native.do_configure attempted to execute
> > unexpectedly
> > ==> default: ERROR: Task python-native.do_compile attempted to execute
> > unexpectedly
> > ==> default: ERROR: Task python-native.do_install attempted to execute
> > unexpectedly
> > ==> default: ERROR: Task python-native.do_populate_sysroot attempted to
> > execute unexpectedly and should have been setscened
> > ==> default: ERROR: SDK preparation failed: error log written to
> > /home/vagrant/poky_sdk/preparing_build_system.log
> >
>
> Basically this means that these tasks tried to execute where really the
> results should have been able to be restored from sstate.
>
> The cause of this type of error is one of three things:
>
> 1) The sstate archive corresponding to a task wasn't able to be fetched
> from
> the server (for a minimal eSDK) or wasn't present in the installer (for a
> full
> eSDK - less likely as we basically do a trial run as part of building the
> eSDK
> in the first place)
>
> 2) The signature was somehow different to what it should have been. (Locked
> signatures are supposed to guard against this.)
>
> 3) A task that wasn't expected to execute did execute and thus the sstate
> wasn't available.
>
> Given that this was python-native which I would expect would be a normal
> part
> of the SDK, I would suspect #1. Is this a minimal or full eSDK (i.e. what
> is
> SDK_EXT_TYPE set to)?
>

That was a "full" eSDK. I noticed that the "same" eSDK installer from
another build host was not affected and I'm trying to rebuild on the
original one with even more recent revision and see if it still happens or
not. Failure with the first one was repeatable, hence I suspect an issue at
sdk population stage, not during installation.

[-- Attachment #2: Type: text/html, Size: 4013 bytes --]

Re: eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 3880 bytes --]

On Wed, Sep 20, 2017 at 11:26 PM, Andrea Galbusera <gizero@gmail.com> wrote:

> Hi Paul,
> thanks for explaining and helping sorting this out.
>
> On Wed, Sep 20, 2017 at 11:54 AM, Paul Eggleton <
> paul.eggleton@linux.intel.com> wrote:
>
>> Hi Andrea,
>>
>> On Wednesday, 20 September 2017 8:44:22 PM NZST Andrea Galbusera wrote:
>> > Seeing the errors below while installing an eSDK. This is a routinely
>> > generated VM that installs the eSDK from installation script. The errors
>> > appeared with the latest iteration of the eSDK script, which is
>> generated
>> > with almost up-to-date revisions from master. Of course I have extra
>> layers
>> > in the mix, but none of them apparently had relevant changed since last
>> > (working) iteration: mostly synching to master branches happened. Can
>> > anyone help suggesting how to investigate this further? What do those
>> > unexpected task mean? I'm blocked on releasing this SDK to developers
>> and
>> > clues from expert would be very appreciated...
>> >
>> > ==> default: Checking sstate mirror object availability...
>> > ==> default: done.
>> > ==> default: ERROR: Task python-native.do_fetch attempted to execute
>> > unexpectedly
>> > ==> default: ERROR: Task python-native.do_prepare_recipe_sysroot
>> attempted
>> > to execute unexpectedly
>> > ==> default: ERROR: Task python-native.do_unpack attempted to execute
>> > unexpectedly
>> > ==> default: ERROR: Task python-native.do_patch attempted to execute
>> > unexpectedly
>> > ==> default: ERROR: Task python-native.do_populate_lic attempted to
>> execute
>> > unexpectedly and should have been setscened
>> > ==> default: ERROR: Task python-native.do_configure attempted to execute
>> > unexpectedly
>> > ==> default: ERROR: Task python-native.do_compile attempted to execute
>> > unexpectedly
>> > ==> default: ERROR: Task python-native.do_install attempted to execute
>> > unexpectedly
>> > ==> default: ERROR: Task python-native.do_populate_sysroot attempted to
>> > execute unexpectedly and should have been setscened
>> > ==> default: ERROR: SDK preparation failed: error log written to
>> > /home/vagrant/poky_sdk/preparing_build_system.log
>> >
>>
>> Basically this means that these tasks tried to execute where really the
>> results should have been able to be restored from sstate.
>>
>> The cause of this type of error is one of three things:
>>
>> 1) The sstate archive corresponding to a task wasn't able to be fetched
>> from
>> the server (for a minimal eSDK) or wasn't present in the installer (for a
>> full
>> eSDK - less likely as we basically do a trial run as part of building the
>> eSDK
>> in the first place)
>>
>> 2) The signature was somehow different to what it should have been.
>> (Locked
>> signatures are supposed to guard against this.)
>>
>> 3) A task that wasn't expected to execute did execute and thus the sstate
>> wasn't available.
>>
>> Given that this was python-native which I would expect would be a normal
>> part
>> of the SDK, I would suspect #1. Is this a minimal or full eSDK (i.e. what
>> is
>> SDK_EXT_TYPE set to)?
>>
>
> That was a "full" eSDK. I noticed that the "same" eSDK installer from
> another build host was not affected and I'm trying to rebuild on the
> original one with even more recent revision and see if it still happens or
> not. Failure with the first one was repeatable, hence I suspect an issue at
> sdk population stage, not during installation.
>

Nuking tmp/ and rebuilding with the same revisions of the successful build
host didn't fix the issue... Same error on installation of the generated
eSDK. Would you suggest any other investigation step? On my todo list I'd
put using the sstate from that other build host than nuking local
sstate-cache/ and going to take a very long coffee break. :-(

[-- Attachment #2: Type: text/html, Size: 4821 bytes --]

Re: eSDK install script failure

[Paul Eggleton]

On Friday, 22 September 2017 1:06:19 AM NZST Andrea Galbusera wrote:
> On Wed, Sep 20, 2017 at 11:26 PM, Andrea Galbusera <gizero@gmail.com> wrote:
> > On Wed, Sep 20, 2017 at 11:54 AM, Paul Eggleton <
> > paul.eggleton@linux.intel.com> wrote:
> >> On Wednesday, 20 September 2017 8:44:22 PM NZST Andrea Galbusera wrote:
> >> > Seeing the errors below while installing an eSDK. This is a routinely
> >> > generated VM that installs the eSDK from installation script. The errors
> >> > appeared with the latest iteration of the eSDK script, which is
> >> generated
> >> > with almost up-to-date revisions from master. Of course I have extra
> >> layers
> >> > in the mix, but none of them apparently had relevant changed since last
> >> > (working) iteration: mostly synching to master branches happened. Can
> >> > anyone help suggesting how to investigate this further? What do those
> >> > unexpected task mean? I'm blocked on releasing this SDK to developers
> >> and
> >> > clues from expert would be very appreciated...
> >> >
> >> > ==> default: Checking sstate mirror object availability...
> >> > ==> default: done.
> >> > ==> default: ERROR: Task python-native.do_fetch attempted to execute
> >> > unexpectedly
> >> > ==> default: ERROR: Task python-native.do_prepare_recipe_sysroot
> >> attempted
> >> > to execute unexpectedly
> >> > ==> default: ERROR: Task python-native.do_unpack attempted to execute
> >> > unexpectedly
> >> > ==> default: ERROR: Task python-native.do_patch attempted to execute
> >> > unexpectedly
> >> > ==> default: ERROR: Task python-native.do_populate_lic attempted to
> >> execute
> >> > unexpectedly and should have been setscened
> >> > ==> default: ERROR: Task python-native.do_configure attempted to execute
> >> > unexpectedly
> >> > ==> default: ERROR: Task python-native.do_compile attempted to execute
> >> > unexpectedly
> >> > ==> default: ERROR: Task python-native.do_install attempted to execute
> >> > unexpectedly
> >> > ==> default: ERROR: Task python-native.do_populate_sysroot attempted to
> >> > execute unexpectedly and should have been setscened
> >> > ==> default: ERROR: SDK preparation failed: error log written to
> >> > /home/vagrant/poky_sdk/preparing_build_system.log
> >> >
> >>
> >> Basically this means that these tasks tried to execute where really the
> >> results should have been able to be restored from sstate.
> >>
> >> The cause of this type of error is one of three things:
> >>
> >> 1) The sstate archive corresponding to a task wasn't able to be fetched
> >> from
> >> the server (for a minimal eSDK) or wasn't present in the installer (for a
> >> full
> >> eSDK - less likely as we basically do a trial run as part of building the
> >> eSDK
> >> in the first place)
> >>
> >> 2) The signature was somehow different to what it should have been.
> >> (Locked
> >> signatures are supposed to guard against this.)
> >>
> >> 3) A task that wasn't expected to execute did execute and thus the sstate
> >> wasn't available.
> >>
> >> Given that this was python-native which I would expect would be a normal
> >> part
> >> of the SDK, I would suspect #1. Is this a minimal or full eSDK (i.e. what
> >> is
> >> SDK_EXT_TYPE set to)?
> >>
> >
> > That was a "full" eSDK. I noticed that the "same" eSDK installer from
> > another build host was not affected and I'm trying to rebuild on the
> > original one with even more recent revision and see if it still happens or
> > not. Failure with the first one was repeatable, hence I suspect an issue at
> > sdk population stage, not during installation.
> 
> Nuking tmp/ and rebuilding with the same revisions of the successful build
> host didn't fix the issue... Same error on installation of the generated
> eSDK. Would you suggest any other investigation step? On my todo list I'd
> put using the sstate from that other build host than nuking local
> sstate-cache/ and going to take a very long coffee break. :-(

Right, so the next step would be looking for the hash for
python-native.do_populate_sysroot in conf/locked_sigs.inc within the failed
SDK install directory and then looking for that in both the sstate-cache
directory within the failed SDK and then in the sstate-cache directory of
the build that built it. I suspect it may not be there, but let me know what
you find.

Cheers,
Paul


-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 5120 bytes --]

On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> On Friday, 22 September 2017 1:06:19 AM NZST Andrea Galbusera wrote:
> > On Wed, Sep 20, 2017 at 11:26 PM, Andrea Galbusera <gizero@gmail.com>
> wrote:
> > > On Wed, Sep 20, 2017 at 11:54 AM, Paul Eggleton <
> > > paul.eggleton@linux.intel.com> wrote:
> > >> On Wednesday, 20 September 2017 8:44:22 PM NZST Andrea Galbusera
> wrote:
> > >> > Seeing the errors below while installing an eSDK. This is a
> routinely
> > >> > generated VM that installs the eSDK from installation script. The
> errors
> > >> > appeared with the latest iteration of the eSDK script, which is
> > >> generated
> > >> > with almost up-to-date revisions from master. Of course I have extra
> > >> layers
> > >> > in the mix, but none of them apparently had relevant changed since
> last
> > >> > (working) iteration: mostly synching to master branches happened.
> Can
> > >> > anyone help suggesting how to investigate this further? What do
> those
> > >> > unexpected task mean? I'm blocked on releasing this SDK to
> developers
> > >> and
> > >> > clues from expert would be very appreciated...
> > >> >
> > >> > ==> default: Checking sstate mirror object availability...
> > >> > ==> default: done.
> > >> > ==> default: ERROR: Task python-native.do_fetch attempted to execute
> > >> > unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_prepare_recipe_sysroot
> > >> attempted
> > >> > to execute unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_unpack attempted to
> execute
> > >> > unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_patch attempted to execute
> > >> > unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_populate_lic attempted to
> > >> execute
> > >> > unexpectedly and should have been setscened
> > >> > ==> default: ERROR: Task python-native.do_configure attempted to
> execute
> > >> > unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_compile attempted to
> execute
> > >> > unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_install attempted to
> execute
> > >> > unexpectedly
> > >> > ==> default: ERROR: Task python-native.do_populate_sysroot
> attempted to
> > >> > execute unexpectedly and should have been setscened
> > >> > ==> default: ERROR: SDK preparation failed: error log written to
> > >> > /home/vagrant/poky_sdk/preparing_build_system.log
> > >> >
> > >>
> > >> Basically this means that these tasks tried to execute where really
> the
> > >> results should have been able to be restored from sstate.
> > >>
> > >> The cause of this type of error is one of three things:
> > >>
> > >> 1) The sstate archive corresponding to a task wasn't able to be
> fetched
> > >> from
> > >> the server (for a minimal eSDK) or wasn't present in the installer
> (for a
> > >> full
> > >> eSDK - less likely as we basically do a trial run as part of building
> the
> > >> eSDK
> > >> in the first place)
> > >>
> > >> 2) The signature was somehow different to what it should have been.
> > >> (Locked
> > >> signatures are supposed to guard against this.)
> > >>
> > >> 3) A task that wasn't expected to execute did execute and thus the
> sstate
> > >> wasn't available.
> > >>
> > >> Given that this was python-native which I would expect would be a
> normal
> > >> part
> > >> of the SDK, I would suspect #1. Is this a minimal or full eSDK (i.e.
> what
> > >> is
> > >> SDK_EXT_TYPE set to)?
> > >>
> > >
> > > That was a "full" eSDK. I noticed that the "same" eSDK installer from
> > > another build host was not affected and I'm trying to rebuild on the
> > > original one with even more recent revision and see if it still
> happens or
> > > not. Failure with the first one was repeatable, hence I suspect an
> issue at
> > > sdk population stage, not during installation.
> >
> > Nuking tmp/ and rebuilding with the same revisions of the successful
> build
> > host didn't fix the issue... Same error on installation of the generated
> > eSDK. Would you suggest any other investigation step? On my todo list I'd
> > put using the sstate from that other build host than nuking local
> > sstate-cache/ and going to take a very long coffee break. :-(
>
> Right, so the next step would be looking for the hash for
> python-native.do_populate_sysroot in conf/locked_sigs.inc within the
> failed
> SDK install directory and then looking for that in both the sstate-cache
> directory within the failed SDK and then in the sstate-cache directory of
> the build that built it. I suspect it may not be there, but let me know
> what
> you find.
>

Good catch! In the failing SDK neither conf/locked_sigs.inc nor
sstate-cache do include any python-native signature or object... Only
python3-native stuff is there. Weird! As said, SDKs from the same build
directory, used to work since a few weeks ago. May any recent change in
poky master have caused this while periodically upgrading without
regenerating the sstate-cache?

[-- Attachment #2: Type: text/html, Size: 6615 bytes --]

Re: eSDK install script failure

[Paul Eggleton]

On Friday, 22 September 2017 9:22:08 AM NZST Andrea Galbusera wrote:
> On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
> paul.eggleton@linux.intel.com> wrote:
> > Right, so the next step would be looking for the hash for
> > python-native.do_populate_sysroot in conf/locked_sigs.inc within the
> > failed SDK install directory and then looking for that in both the sstate-
> > cache directory within the failed SDK and then in the sstate-cache
> > directory of the build that built it. I suspect it may not be there, but
> > let me know what you find.
> 
> Good catch! In the failing SDK neither conf/locked_sigs.inc nor
> sstate-cache do include any python-native signature or object... Only
> python3-native stuff is there. Weird! As said, SDKs from the same build
> directory, used to work since a few weeks ago. May any recent change in
> poky master have caused this while periodically upgrading without
> regenerating the sstate-cache?

No, I can't see any added references to python-native anywhere in the last few 
weeks. If you do bitbake -c clean python-native and then rebuild the SDK does 
the issue go away?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: eSDK install script failure

[Paul Eggleton]

On Friday, 22 September 2017 9:40:41 AM NZST Paul Eggleton wrote:
> On Friday, 22 September 2017 9:22:08 AM NZST Andrea Galbusera wrote:
> > On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
> > paul.eggleton@linux.intel.com> wrote:
> > > Right, so the next step would be looking for the hash for
> > > python-native.do_populate_sysroot in conf/locked_sigs.inc within the
> > > failed SDK install directory and then looking for that in both the sstate-
> > > cache directory within the failed SDK and then in the sstate-cache
> > > directory of the build that built it. I suspect it may not be there, but
> > > let me know what you find.
> > 
> > Good catch! In the failing SDK neither conf/locked_sigs.inc nor
> > sstate-cache do include any python-native signature or object... Only
> > python3-native stuff is there. Weird! As said, SDKs from the same build
> > directory, used to work since a few weeks ago. May any recent change in
> > poky master have caused this while periodically upgrading without
> > regenerating the sstate-cache?
> 
> No, I can't see any added references to python-native anywhere in the last few 
> weeks. If you do bitbake -c clean python-native and then rebuild the SDK does 
> the issue go away?

Actually scratch that, that's not going to help. The question is where is
this dependency coming from and why isn't it properly picked up such
that it gets included. bitbake -g -c populate_sdk_ext your-image might be 
useful in determining that.

Cheers,
Paul


-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 8550 bytes --]

Hi Paul,

I'm back to this issue after the weeken break. See below the feedback from
your suggestions.

On Thu, Sep 21, 2017 at 11:49 PM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> On Friday, 22 September 2017 9:40:41 AM NZST Paul Eggleton wrote:
> > On Friday, 22 September 2017 9:22:08 AM NZST Andrea Galbusera wrote:
> > > On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
> > > paul.eggleton@linux.intel.com> wrote:
> > > > Right, so the next step would be looking for the hash for
> > > > python-native.do_populate_sysroot in conf/locked_sigs.inc within the
> > > > failed SDK install directory and then looking for that in both the
> sstate-
> > > > cache directory within the failed SDK and then in the sstate-cache
> > > > directory of the build that built it. I suspect it may not be there,
> but
> > > > let me know what you find.
> > >
> > > Good catch! In the failing SDK neither conf/locked_sigs.inc nor
> > > sstate-cache do include any python-native signature or object... Only
> > > python3-native stuff is there. Weird! As said, SDKs from the same build
> > > directory, used to work since a few weeks ago. May any recent change in
> > > poky master have caused this while periodically upgrading without
> > > regenerating the sstate-cache?
> >
> > No, I can't see any added references to python-native anywhere in the
> last few
> > weeks. If you do bitbake -c clean python-native and then rebuild the SDK
> does
> > the issue go away?
>
> Actually scratch that, that's not going to help. The question is where is
> this dependency coming from and why isn't it properly picked up such
> that it gets included. bitbake -g -c populate_sdk_ext your-image might be
> useful in determining that.
>

$ bitbake core-image-base-dlms -c populate_sdk_ext -g

Grepping task-depends.dot for "python-native" gives no match. Surprisingly
enough (at least for me) I read a different story when doing the same for
the image itself

$ bitbake core-image-base-dlms -g
$ grep python-native task-depends.dot
 "python-native.do_populate_lic" [label="python-native
do_populate_lic\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python[18/7956]
.7.13.bb"]
"python-native.do_populate_lic" -> "python-native.do_patch"
"python-native.do_prepare_recipe_sysroot" [label="python-native
do_prepare_recipe_sysroot\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/py
thon/python-native_2.7.13.bb"]
"python-native.do_prepare_recipe_sysroot" ->
"openssl-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"pkgconfig-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"automake-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"expat-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"sqlite3-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" -> "python-native.do_fetch"
"python-native.do_prepare_recipe_sysroot" ->
"bzip2-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"readline-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"zlib-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"autoconf-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"gnu-config-native.do_populate_sysroot"
"python-native.do_prepare_recipe_sysroot" ->
"libtool-native.do_populate_sysroot"
"python-native.do_rm_work_all" [label="python-native
do_rm_work_all\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python-native_2.7
.13.bb"]
"python-native.do_rm_work_all" -> "readline-native.do_rm_work"
"python-native.do_rm_work_all" -> "gnu-config-native.do_rm_work"
"python-native.do_rm_work_all" -> "openssl-native.do_rm_work"
"python-native.do_rm_work_all" -> "automake-native.do_rm_work"
"python-native.do_rm_work_all" -> "m4-native.do_rm_work"
"python-native.do_rm_work_all" -> "makedepend-native.do_rm_work"
"python-native.do_rm_work_all" -> "xproto-native.do_rm_work"
"python-native.do_rm_work_all" -> "bzip2-native.do_rm_work"
"python-native.do_rm_work_all" -> "ncurses-native.do_rm_work"
"python-native.do_rm_work_all" -> "python-native.do_rm_work"
"python-native.do_rm_work_all" -> "expat-native.do_rm_work"
"python-native.do_rm_work_all" -> "pigz-native.do_rm_work"
"python-native.do_rm_work_all" -> "libtool-native.do_rm_work"
"python-native.do_rm_work_all" -> "pkgconfig-native.do_rm_work"
"python-native.do_rm_work_all" -> "gettext-minimal-native.do_rm_work"
"python-native.do_rm_work_all" -> "util-macros-native.do_rm_work"
"python-native.do_rm_work_all" -> "quilt-native.do_rm_work"
"python-native.do_rm_work_all" -> "autoconf-native.do_rm_work"
"python-native.do_rm_work_all" -> "cryptodev-linux-native.do_rm_work"
"python-native.do_rm_work_all" -> "xz-native.do_rm_work"
"python-native.do_rm_work_all" -> "zlib-native.do_rm_work"
"python-native.do_rm_work_all" -> "texinfo-dummy-native.do_rm_work"
"python-native.do_rm_work_all" -> "sqlite3-native.do_rm_work"
"python-native.do_compile" [label="python-native
do_compile\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_compile" -> "python-native.do_configure"
"python-native.do_install" [label="python-native
do_install\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_install" -> "python-native.do_compile"
"bmap-tools-native.do_rm_work_all" -> "python-native.do_rm_work"
"python-native.do_build" [label="python-native
do_build\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_build" -> "python-native.do_populate_sysroot"
"python-native.do_build" -> "python-native.do_rm_work"
"python-native.do_build" -> "python-native.do_populate_lic"
"python-native.do_build" -> "python-native.do_rm_work_all"
"python-native.do_fetch" [label="python-native
do_fetch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_unpack" [label="python-native
do_unpack\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_unpack" -> "xz-native.do_populate_sysroot"
"python-native.do_unpack" -> "python-native.do_fetch"
"python-native.do_patch" [label="python-native
do_patch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_patch" -> "python-native.do_unpack"
"python-native.do_patch" -> "quilt-native.do_populate_sysroot"
"core-image-base-dlms.do_build" -> "python-native.do_build"
"python-native.do_configure" [label="python-native
do_configure\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python-native_2.7.13.
bb"]
"python-native.do_configure" -> "python-native.do_patch"
"python-native.do_configure" -> "python-native.do_prepare_recipe_sysroot"
"python-native.do_rm_work" [label="python-native
do_rm_work\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
python-native_2.7.13.bb"]
"python-native.do_rm_work" -> "python-native.do_populate_sysroot"
"python-native.do_rm_work" -> "python-native.do_populate_lic"
"python-native.do_populate_sysroot" [label="python-native
do_populate_sysroot\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python-
native_2.7.13.bb"]
"python-native.do_populate_sysroot" -> "python-native.do_install"

As expected, on the build bot that builds successfully installable SDKs,
both dependency graphs contain no mention of python-native. Time to nuke
the sstate-cache from the failing one or do you have any additional clue to
investigate further?

[-- Attachment #2: Type: text/html, Size: 11293 bytes --]

Re: eSDK install script failure

[Paul Eggleton]

Hi Andrea,

On Tuesday, 26 September 2017 4:33:46 AM NZDT Andrea Galbusera wrote:
> I'm back to this issue after the weeken break. See below the feedback from
> your suggestions.
> 
> On Thu, Sep 21, 2017 at 11:49 PM, Paul Eggleton <
> paul.eggleton@linux.intel.com> wrote:
> 
> > On Friday, 22 September 2017 9:40:41 AM NZST Paul Eggleton wrote:
> > > On Friday, 22 September 2017 9:22:08 AM NZST Andrea Galbusera wrote:
> > > > On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
> > > > paul.eggleton@linux.intel.com> wrote:
> > > > > Right, so the next step would be looking for the hash for
> > > > > python-native.do_populate_sysroot in conf/locked_sigs.inc within the
> > > > > failed SDK install directory and then looking for that in both the
> > sstate-
> > > > > cache directory within the failed SDK and then in the sstate-cache
> > > > > directory of the build that built it. I suspect it may not be there,
> > but
> > > > > let me know what you find.
> > > >
> > > > Good catch! In the failing SDK neither conf/locked_sigs.inc nor
> > > > sstate-cache do include any python-native signature or object... Only
> > > > python3-native stuff is there. Weird! As said, SDKs from the same build
> > > > directory, used to work since a few weeks ago. May any recent change in
> > > > poky master have caused this while periodically upgrading without
> > > > regenerating the sstate-cache?
> > >
> > > No, I can't see any added references to python-native anywhere in the
> > last few
> > > weeks. If you do bitbake -c clean python-native and then rebuild the SDK
> > does
> > > the issue go away?
> >
> > Actually scratch that, that's not going to help. The question is where is
> > this dependency coming from and why isn't it properly picked up such
> > that it gets included. bitbake -g -c populate_sdk_ext your-image might be
> > useful in determining that.
> >
> 
> $ bitbake core-image-base-dlms -c populate_sdk_ext -g
> 
> Grepping task-depends.dot for "python-native" gives no match. Surprisingly
> enough (at least for me) I read a different story when doing the same for
> the image itself
> 
> $ bitbake core-image-base-dlms -g
> $ grep python-native task-depends.dot
>  "python-native.do_populate_lic" [label="python-native
> do_populate_lic\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python[18/7956]
> .7.13.bb"]
> "python-native.do_populate_lic" -> "python-native.do_patch"
> "python-native.do_prepare_recipe_sysroot" [label="python-native
> do_prepare_recipe_sysroot\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/py
> thon/python-native_2.7.13.bb"]
> "python-native.do_prepare_recipe_sysroot" ->
> "openssl-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "pkgconfig-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "automake-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "expat-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "sqlite3-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" -> "python-native.do_fetch"
> "python-native.do_prepare_recipe_sysroot" ->
> "bzip2-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "readline-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "zlib-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "autoconf-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "gnu-config-native.do_populate_sysroot"
> "python-native.do_prepare_recipe_sysroot" ->
> "libtool-native.do_populate_sysroot"
> "python-native.do_rm_work_all" [label="python-native
> do_rm_work_all\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python-native_2.7
> .13.bb"]
> "python-native.do_rm_work_all" -> "readline-native.do_rm_work"
> "python-native.do_rm_work_all" -> "gnu-config-native.do_rm_work"
> "python-native.do_rm_work_all" -> "openssl-native.do_rm_work"
> "python-native.do_rm_work_all" -> "automake-native.do_rm_work"
> "python-native.do_rm_work_all" -> "m4-native.do_rm_work"
> "python-native.do_rm_work_all" -> "makedepend-native.do_rm_work"
> "python-native.do_rm_work_all" -> "xproto-native.do_rm_work"
> "python-native.do_rm_work_all" -> "bzip2-native.do_rm_work"
> "python-native.do_rm_work_all" -> "ncurses-native.do_rm_work"
> "python-native.do_rm_work_all" -> "python-native.do_rm_work"
> "python-native.do_rm_work_all" -> "expat-native.do_rm_work"
> "python-native.do_rm_work_all" -> "pigz-native.do_rm_work"
> "python-native.do_rm_work_all" -> "libtool-native.do_rm_work"
> "python-native.do_rm_work_all" -> "pkgconfig-native.do_rm_work"
> "python-native.do_rm_work_all" -> "gettext-minimal-native.do_rm_work"
> "python-native.do_rm_work_all" -> "util-macros-native.do_rm_work"
> "python-native.do_rm_work_all" -> "quilt-native.do_rm_work"
> "python-native.do_rm_work_all" -> "autoconf-native.do_rm_work"
> "python-native.do_rm_work_all" -> "cryptodev-linux-native.do_rm_work"
> "python-native.do_rm_work_all" -> "xz-native.do_rm_work"
> "python-native.do_rm_work_all" -> "zlib-native.do_rm_work"
> "python-native.do_rm_work_all" -> "texinfo-dummy-native.do_rm_work"
> "python-native.do_rm_work_all" -> "sqlite3-native.do_rm_work"
> "python-native.do_compile" [label="python-native
> do_compile\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_compile" -> "python-native.do_configure"
> "python-native.do_install" [label="python-native
> do_install\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_install" -> "python-native.do_compile"
> "bmap-tools-native.do_rm_work_all" -> "python-native.do_rm_work"
> "python-native.do_build" [label="python-native
> do_build\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_build" -> "python-native.do_populate_sysroot"
> "python-native.do_build" -> "python-native.do_rm_work"
> "python-native.do_build" -> "python-native.do_populate_lic"
> "python-native.do_build" -> "python-native.do_rm_work_all"
> "python-native.do_fetch" [label="python-native
> do_fetch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_unpack" [label="python-native
> do_unpack\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_unpack" -> "xz-native.do_populate_sysroot"
> "python-native.do_unpack" -> "python-native.do_fetch"
> "python-native.do_patch" [label="python-native
> do_patch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_patch" -> "python-native.do_unpack"
> "python-native.do_patch" -> "quilt-native.do_populate_sysroot"
> "core-image-base-dlms.do_build" -> "python-native.do_build"
> "python-native.do_configure" [label="python-native
> do_configure\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python-native_2.7.13.
> bb"]
> "python-native.do_configure" -> "python-native.do_patch"
> "python-native.do_configure" -> "python-native.do_prepare_recipe_sysroot"
> "python-native.do_rm_work" [label="python-native
> do_rm_work\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/
> python-native_2.7.13.bb"]
> "python-native.do_rm_work" -> "python-native.do_populate_sysroot"
> "python-native.do_rm_work" -> "python-native.do_populate_lic"
> "python-native.do_populate_sysroot" [label="python-native
> do_populate_sysroot\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-master/build-poky/conf/../../layers/poky/meta/recipes-devtools/python/python-
> native_2.7.13.bb"]
> "python-native.do_populate_sysroot" -> "python-native.do_install"
> 
> As expected, on the build bot that builds successfully installable SDKs,
> both dependency graphs contain no mention of python-native. Time to nuke
> the sstate-cache from the failing one or do you have any additional clue to
> investigate further?

Deleting sstate-cache is very unlikely to help - these dependency relationships
aren't stored in sstate (or at least they aren't stored there in a way that
bitbake will take it as input for dependency graph generation). 

It looks like bmap-tools-native might be how the dependency is getting in -
perhaps this is coming in conditionally through IMAGE_FSTYPES? For some 
unknown reason it also doesn't get triggered when building the eSDK but does
when restoring the image dependencies within the eSDK.

Cheers,
Paul


-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 10824 bytes --]

Paul,

On Tue, Sep 26, 2017 at 6:33 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> Hi Andrea,
>
> On Tuesday, 26 September 2017 4:33:46 AM NZDT Andrea Galbusera wrote:
> > I'm back to this issue after the weeken break. See below the feedback
> from
> > your suggestions.
> >
> > On Thu, Sep 21, 2017 at 11:49 PM, Paul Eggleton <
> > paul.eggleton@linux.intel.com> wrote:
> >
> > > On Friday, 22 September 2017 9:40:41 AM NZST Paul Eggleton wrote:
> > > > On Friday, 22 September 2017 9:22:08 AM NZST Andrea Galbusera wrote:
> > > > > On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
> > > > > paul.eggleton@linux.intel.com> wrote:
> > > > > > Right, so the next step would be looking for the hash for
> > > > > > python-native.do_populate_sysroot in conf/locked_sigs.inc
> within the
> > > > > > failed SDK install directory and then looking for that in both
> the
> > > sstate-
> > > > > > cache directory within the failed SDK and then in the
> sstate-cache
> > > > > > directory of the build that built it. I suspect it may not be
> there,
> > > but
> > > > > > let me know what you find.
> > > > >
> > > > > Good catch! In the failing SDK neither conf/locked_sigs.inc nor
> > > > > sstate-cache do include any python-native signature or object...
> Only
> > > > > python3-native stuff is there. Weird! As said, SDKs from the same
> build
> > > > > directory, used to work since a few weeks ago. May any recent
> change in
> > > > > poky master have caused this while periodically upgrading without
> > > > > regenerating the sstate-cache?
> > > >
> > > > No, I can't see any added references to python-native anywhere in the
> > > last few
> > > > weeks. If you do bitbake -c clean python-native and then rebuild the
> SDK
> > > does
> > > > the issue go away?
> > >
> > > Actually scratch that, that's not going to help. The question is where
> is
> > > this dependency coming from and why isn't it properly picked up such
> > > that it gets included. bitbake -g -c populate_sdk_ext your-image might
> be
> > > useful in determining that.
> > >
> >
> > $ bitbake core-image-base-dlms -c populate_sdk_ext -g
> >
> > Grepping task-depends.dot for "python-native" gives no match.
> Surprisingly
> > enough (at least for me) I read a different story when doing the same for
> > the image itself
> >
> > $ bitbake core-image-base-dlms -g
> > $ grep python-native task-depends.dot
> >  "python-native.do_populate_lic" [label="python-native
> > do_populate_lic\n:2.7.13-r1.1\n/home/gizero/work/
> smartliving/distro/repo-master/build-poky/conf/../../
> layers/poky/meta/recipes-devtools/python/python[18/7956]
> > .7.13.bb"]
> > "python-native.do_populate_lic" -> "python-native.do_patch"
> > "python-native.do_prepare_recipe_sysroot" [label="python-native
> > do_prepare_recipe_sysroot\n:2.7.13-r1.1\n/home/gizero/work/
> smartliving/distro/repo-master/build-poky/conf/../../
> layers/poky/meta/recipes-devtools/py
> > thon/python-native_2.7.13.bb"]
> > "python-native.do_prepare_recipe_sysroot" ->
> > "openssl-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "pkgconfig-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "automake-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "expat-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "sqlite3-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" -> "python-native.do_fetch"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "bzip2-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "readline-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "zlib-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "autoconf-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "gnu-config-native.do_populate_sysroot"
> > "python-native.do_prepare_recipe_sysroot" ->
> > "libtool-native.do_populate_sysroot"
> > "python-native.do_rm_work_all" [label="python-native
> > do_rm_work_all\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro/repo-
> master/build-poky/conf/../../layers/poky/meta/recipes-
> devtools/python/python-native_2.7
> > .13.bb"]
> > "python-native.do_rm_work_all" -> "readline-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "gnu-config-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "openssl-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "automake-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "m4-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "makedepend-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "xproto-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "bzip2-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "ncurses-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "python-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "expat-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "pigz-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "libtool-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "pkgconfig-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "gettext-minimal-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "util-macros-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "quilt-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "autoconf-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "cryptodev-linux-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "xz-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "zlib-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "texinfo-dummy-native.do_rm_work"
> > "python-native.do_rm_work_all" -> "sqlite3-native.do_rm_work"
> > "python-native.do_compile" [label="python-native
> > do_compile\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_compile" -> "python-native.do_configure"
> > "python-native.do_install" [label="python-native
> > do_install\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_install" -> "python-native.do_compile"
> > "bmap-tools-native.do_rm_work_all" -> "python-native.do_rm_work"
> > "python-native.do_build" [label="python-native
> > do_build\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_build" -> "python-native.do_populate_sysroot"
> > "python-native.do_build" -> "python-native.do_rm_work"
> > "python-native.do_build" -> "python-native.do_populate_lic"
> > "python-native.do_build" -> "python-native.do_rm_work_all"
> > "python-native.do_fetch" [label="python-native
> > do_fetch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_unpack" [label="python-native
> > do_unpack\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_unpack" -> "xz-native.do_populate_sysroot"
> > "python-native.do_unpack" -> "python-native.do_fetch"
> > "python-native.do_patch" [label="python-native
> > do_patch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_patch" -> "python-native.do_unpack"
> > "python-native.do_patch" -> "quilt-native.do_populate_sysroot"
> > "core-image-base-dlms.do_build" -> "python-native.do_build"
> > "python-native.do_configure" [label="python-native
> > do_configure\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/python-native_2.7.13.
> > bb"]
> > "python-native.do_configure" -> "python-native.do_patch"
> > "python-native.do_configure" -> "python-native.do_prepare_
> recipe_sysroot"
> > "python-native.do_rm_work" [label="python-native
> > do_rm_work\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
> distro/repo-master/build-poky/conf/../../layers/poky/meta/
> recipes-devtools/python/
> > python-native_2.7.13.bb"]
> > "python-native.do_rm_work" -> "python-native.do_populate_sysroot"
> > "python-native.do_rm_work" -> "python-native.do_populate_lic"
> > "python-native.do_populate_sysroot" [label="python-native
> > do_populate_sysroot\n:2.7.13-r1.1\n/home/gizero/work/
> smartliving/distro/repo-master/build-poky/conf/../../
> layers/poky/meta/recipes-devtools/python/python-
> > native_2.7.13.bb"]
> > "python-native.do_populate_sysroot" -> "python-native.do_install"
> >
> > As expected, on the build bot that builds successfully installable SDKs,
> > both dependency graphs contain no mention of python-native. Time to nuke
> > the sstate-cache from the failing one or do you have any additional clue
> to
> > investigate further?
>
> Deleting sstate-cache is very unlikely to help - these dependency
> relationships
> aren't stored in sstate (or at least they aren't stored there in a way that
> bitbake will take it as input for dependency graph generation).
>
> It looks like bmap-tools-native might be how the dependency is getting in -
> perhaps this is coming in conditionally through IMAGE_FSTYPES? For some
> unknown reason it also doesn't get triggered when building the eSDK but
> does
> when restoring the image dependencies within the eSDK.
>

Good catch! This explains why the two build systems behaves differently: a
little change in local.conf I didn't notice before... The "failing" system
does IMAGE_FSTYPES_append = " wic", likely bringing bmap-tools-native in,
while the "working" build bot doesn't. However, as you say, this still does
not explain why the eSDK is missing the python-native bits. BTW, "wic" is
in the IMAGE_FSTYPE of my usual build host since months and I'm pretty sure
it did build many successfully installable eSDK with that option in place:
but maybe other tasks which also depended on python-native got removed
along the way...
What I can do now is adding "wic" to IMAGE_FSTYPES on the other system and
see if I get the two scenarios equally misbehaving or not. Anything else or
different you can suggest to nail this down?

[-- Attachment #2: Type: text/html, Size: 15085 bytes --]

Re: eSDK install script failure

[Andrea Galbusera]

[-- Attachment #1: Type: text/plain, Size: 11483 bytes --]

Hi Paul,

On Tue, Sep 26, 2017 at 7:18 AM, Andrea Galbusera <gizero@gmail.com> wrote:

> Paul,
>
> On Tue, Sep 26, 2017 at 6:33 AM, Paul Eggleton <
> paul.eggleton@linux.intel.com> wrote:
>
>> Hi Andrea,
>>
>> On Tuesday, 26 September 2017 4:33:46 AM NZDT Andrea Galbusera wrote:
>> > I'm back to this issue after the weeken break. See below the feedback
>> from
>> > your suggestions.
>> >
>> > On Thu, Sep 21, 2017 at 11:49 PM, Paul Eggleton <
>> > paul.eggleton@linux.intel.com> wrote:
>> >
>> > > On Friday, 22 September 2017 9:40:41 AM NZST Paul Eggleton wrote:
>> > > > On Friday, 22 September 2017 9:22:08 AM NZST Andrea Galbusera wrote:
>> > > > > On Thu, Sep 21, 2017 at 10:48 PM, Paul Eggleton <
>> > > > > paul.eggleton@linux.intel.com> wrote:
>> > > > > > Right, so the next step would be looking for the hash for
>> > > > > > python-native.do_populate_sysroot in conf/locked_sigs.inc
>> within the
>> > > > > > failed SDK install directory and then looking for that in both
>> the
>> > > sstate-
>> > > > > > cache directory within the failed SDK and then in the
>> sstate-cache
>> > > > > > directory of the build that built it. I suspect it may not be
>> there,
>> > > but
>> > > > > > let me know what you find.
>> > > > >
>> > > > > Good catch! In the failing SDK neither conf/locked_sigs.inc nor
>> > > > > sstate-cache do include any python-native signature or object...
>> Only
>> > > > > python3-native stuff is there. Weird! As said, SDKs from the same
>> build
>> > > > > directory, used to work since a few weeks ago. May any recent
>> change in
>> > > > > poky master have caused this while periodically upgrading without
>> > > > > regenerating the sstate-cache?
>> > > >
>> > > > No, I can't see any added references to python-native anywhere in
>> the
>> > > last few
>> > > > weeks. If you do bitbake -c clean python-native and then rebuild
>> the SDK
>> > > does
>> > > > the issue go away?
>> > >
>> > > Actually scratch that, that's not going to help. The question is
>> where is
>> > > this dependency coming from and why isn't it properly picked up such
>> > > that it gets included. bitbake -g -c populate_sdk_ext your-image
>> might be
>> > > useful in determining that.
>> > >
>> >
>> > $ bitbake core-image-base-dlms -c populate_sdk_ext -g
>> >
>> > Grepping task-depends.dot for "python-native" gives no match.
>> Surprisingly
>> > enough (at least for me) I read a different story when doing the same
>> for
>> > the image itself
>> >
>> > $ bitbake core-image-base-dlms -g
>> > $ grep python-native task-depends.dot
>> >  "python-native.do_populate_lic" [label="python-native
>> > do_populate_lic\n:2.7.13-r1.1\n/home/gizero/work/smartliving
>> /distro/repo-master/build-poky/conf/../../layers/poky/meta/r
>> ecipes-devtools/python/python[18/7956]
>> > .7.13.bb"]
>> > "python-native.do_populate_lic" -> "python-native.do_patch"
>> > "python-native.do_prepare_recipe_sysroot" [label="python-native
>> > do_prepare_recipe_sysroot\n:2.7.13-r1.1\n/home/gizero/work/s
>> martliving/distro/repo-master/build-poky/conf/../../layers/p
>> oky/meta/recipes-devtools/py
>> > thon/python-native_2.7.13.bb"]
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "openssl-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "pkgconfig-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "automake-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "expat-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "sqlite3-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" -> "python-native.do_fetch"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "bzip2-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "readline-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "zlib-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "autoconf-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "gnu-config-native.do_populate_sysroot"
>> > "python-native.do_prepare_recipe_sysroot" ->
>> > "libtool-native.do_populate_sysroot"
>> > "python-native.do_rm_work_all" [label="python-native
>> > do_rm_work_all\n:2.7.13-r1.1\n/home/gizero/work/smartliving/
>> distro/repo-master/build-poky/conf/../../layers/poky/meta/re
>> cipes-devtools/python/python-native_2.7
>> > .13.bb"]
>> > "python-native.do_rm_work_all" -> "readline-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "gnu-config-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "openssl-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "automake-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "m4-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "makedepend-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "xproto-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "bzip2-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "ncurses-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "python-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "expat-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "pigz-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "libtool-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "pkgconfig-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "gettext-minimal-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "util-macros-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "quilt-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "autoconf-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "cryptodev-linux-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "xz-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "zlib-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "texinfo-dummy-native.do_rm_work"
>> > "python-native.do_rm_work_all" -> "sqlite3-native.do_rm_work"
>> > "python-native.do_compile" [label="python-native
>> > do_compile\n:2.7.13-r1.1\n/home/gizero/work/smartliving/dist
>> ro/repo-master/build-poky/conf/../../layers/poky/meta/recipe
>> s-devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_compile" -> "python-native.do_configure"
>> > "python-native.do_install" [label="python-native
>> > do_install\n:2.7.13-r1.1\n/home/gizero/work/smartliving/dist
>> ro/repo-master/build-poky/conf/../../layers/poky/meta/recipe
>> s-devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_install" -> "python-native.do_compile"
>> > "bmap-tools-native.do_rm_work_all" -> "python-native.do_rm_work"
>> > "python-native.do_build" [label="python-native
>> > do_build\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro
>> /repo-master/build-poky/conf/../../layers/poky/meta/recipes-
>> devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_build" -> "python-native.do_populate_sysroot"
>> > "python-native.do_build" -> "python-native.do_rm_work"
>> > "python-native.do_build" -> "python-native.do_populate_lic"
>> > "python-native.do_build" -> "python-native.do_rm_work_all"
>> > "python-native.do_fetch" [label="python-native
>> > do_fetch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro
>> /repo-master/build-poky/conf/../../layers/poky/meta/recipes-
>> devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_unpack" [label="python-native
>> > do_unpack\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distr
>> o/repo-master/build-poky/conf/../../layers/poky/meta/recipes
>> -devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_unpack" -> "xz-native.do_populate_sysroot"
>> > "python-native.do_unpack" -> "python-native.do_fetch"
>> > "python-native.do_patch" [label="python-native
>> > do_patch\n:2.7.13-r1.1\n/home/gizero/work/smartliving/distro
>> /repo-master/build-poky/conf/../../layers/poky/meta/recipes-
>> devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_patch" -> "python-native.do_unpack"
>> > "python-native.do_patch" -> "quilt-native.do_populate_sysroot"
>> > "core-image-base-dlms.do_build" -> "python-native.do_build"
>> > "python-native.do_configure" [label="python-native
>> > do_configure\n:2.7.13-r1.1\n/home/gizero/work/smartliving/di
>> stro/repo-master/build-poky/conf/../../layers/poky/meta/reci
>> pes-devtools/python/python-native_2.7.13.
>> > bb"]
>> > "python-native.do_configure" -> "python-native.do_patch"
>> > "python-native.do_configure" -> "python-native.do_prepare_reci
>> pe_sysroot"
>> > "python-native.do_rm_work" [label="python-native
>> > do_rm_work\n:2.7.13-r1.1\n/home/gizero/work/smartliving/dist
>> ro/repo-master/build-poky/conf/../../layers/poky/meta/recipe
>> s-devtools/python/
>> > python-native_2.7.13.bb"]
>> > "python-native.do_rm_work" -> "python-native.do_populate_sysroot"
>> > "python-native.do_rm_work" -> "python-native.do_populate_lic"
>> > "python-native.do_populate_sysroot" [label="python-native
>> > do_populate_sysroot\n:2.7.13-r1.1\n/home/gizero/work/smartli
>> ving/distro/repo-master/build-poky/conf/../../layers/poky/me
>> ta/recipes-devtools/python/python-
>> > native_2.7.13.bb"]
>> > "python-native.do_populate_sysroot" -> "python-native.do_install"
>> >
>> > As expected, on the build bot that builds successfully installable SDKs,
>> > both dependency graphs contain no mention of python-native. Time to nuke
>> > the sstate-cache from the failing one or do you have any additional
>> clue to
>> > investigate further?
>>
>> Deleting sstate-cache is very unlikely to help - these dependency
>> relationships
>> aren't stored in sstate (or at least they aren't stored there in a way
>> that
>> bitbake will take it as input for dependency graph generation).
>>
>> It looks like bmap-tools-native might be how the dependency is getting in
>> -
>> perhaps this is coming in conditionally through IMAGE_FSTYPES? For some
>> unknown reason it also doesn't get triggered when building the eSDK but
>> does
>> when restoring the image dependencies within the eSDK.
>>
>
> Good catch! This explains why the two build systems behaves differently: a
> little change in local.conf I didn't notice before... The "failing" system
> does IMAGE_FSTYPES_append = " wic", likely bringing bmap-tools-native in,
> while the "working" build bot doesn't. However, as you say, this still does
> not explain why the eSDK is missing the python-native bits. BTW, "wic" is
> in the IMAGE_FSTYPE of my usual build host since months and I'm pretty sure
> it did build many successfully installable eSDK with that option in place:
> but maybe other tasks which also depended on python-native got removed
> along the way...
> What I can do now is adding "wic" to IMAGE_FSTYPES on the other system and
> see if I get the two scenarios equally misbehaving or not. Anything else or
> different you can suggest to nail this down?
>

Just noticed this thread [1] on the list and the corresponding bug [2]. Do
you think this issue and the eSDKs with missing python-native can relate to
each other?

[1] https://lists.yoctoproject.org/pipermail/yocto/2017-September/038199.
html
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=12148

[-- Attachment #2: Type: text/html, Size: 16342 bytes --]