Should changing a task in the .bb file cause the task to be rerun?

Should changing a task in the .bb file cause the task to be rerun?

[Sean McKay]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

Hi all,

This is probably a fairly short question (I hope):
I'm working on a branch based on zeus. I found a bug in the way that one of our recipes was handling something during do_configure (which caused an error to pop up in a do_compile task). When I modified the do_configure task (do_configure_append, actually) to behave properly (which involved changing the actual commands run in that function) and reran bitbake -c compile <recipe>, the do_configure task wasn't rerun despite the change to the function's code.

Is it safe to assume this means we've done something to mess up the way our system is processing things? Or is that expected behavior? If the latter (which I don't really expect), is there an explanation somewhere for why changing the commands in a task doesn't prompt bitbake to rerun the task?

Cheers!
-Sean McKay

[-- Attachment #2: Type: text/html, Size: 2638 bytes --]

Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

[Richard Purdie]

On Fri, 2020-02-28 at 22:51 +0000, Sean McKay wrote:
> This is probably a fairly short question (I hope):
> I’m working on a branch based on zeus. I found a bug in the way that
> one of our recipes was handling something during do_configure (which
> caused an error to pop up in a do_compile task). When I modified the
> do_configure task (do_configure_append, actually) to behave properly
> (which involved changing the actual commands run in that function)
> and reran bitbake -c compile <recipe>, the do_configure task wasn’t
> rerun despite the change to the function’s code.
>  
> Is it safe to assume this means we’ve done something to mess up the
> way our system is processing things? Or is that expected behavior

Its not expected behaviour and yes, it sounds like something is messed
up...

Cheers,

Richard

Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

[Mikko Rapeli]

On Fri, Feb 28, 2020 at 11:25:59PM +0000, Richard Purdie wrote:
> On Fri, 2020-02-28 at 22:51 +0000, Sean McKay wrote:
> > This is probably a fairly short question (I hope):
> > I’m working on a branch based on zeus. I found a bug in the way that
> > one of our recipes was handling something during do_configure (which
> > caused an error to pop up in a do_compile task). When I modified the
> > do_configure task (do_configure_append, actually) to behave properly
> > (which involved changing the actual commands run in that function)
> > and reran bitbake -c compile <recipe>, the do_configure task wasn’t
> > rerun despite the change to the function’s code.
> >  
> > Is it safe to assume this means we’ve done something to mess up the
> > way our system is processing things? Or is that expected behavior
> 
> Its not expected behaviour and yes, it sounds like something is messed
> up...

This is what I've come to expect with yocto 2.0 jethro, 2.5 sumo and
3.0 zeus.

That's why I always write:

$ bitbake -c clean recipe && bitbake -c cleansstate && bitbake -c compile recipe

when debugging changes in recipes to_compile and other dependent tasks.
I'm also cleaning up sstate to be sure it's not corrupt or filled with
somehow bad data.

-Mikko

Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

[Richard Purdie]

On Tue, 2020-03-10 at 11:33 +0000, Mikko.Rapeli@bmw.de wrote:
> On Fri, Feb 28, 2020 at 11:25:59PM +0000, Richard Purdie wrote:
> > On Fri, 2020-02-28 at 22:51 +0000, Sean McKay wrote:
> > > This is probably a fairly short question (I hope):
> > > I’m working on a branch based on zeus. I found a bug in the way that
> > > one of our recipes was handling something during do_configure (which
> > > caused an error to pop up in a do_compile task). When I modified the
> > > do_configure task (do_configure_append, actually) to behave properly
> > > (which involved changing the actual commands run in that function)
> > > and reran bitbake -c compile <recipe>, the do_configure task wasn’t
> > > rerun despite the change to the function’s code.
> > >  
> > > Is it safe to assume this means we’ve done something to mess up the
> > > way our system is processing things? Or is that expected behavior
> > 
> > Its not expected behaviour and yes, it sounds like something is messed
> > up...
> 
> This is what I've come to expect with yocto 2.0 jethro, 2.5 sumo and
> 3.0 zeus.
> 
> That's why I always write:
> 
> $ bitbake -c clean recipe && bitbake -c cleansstate && bitbake -c compile recipe
> 
> when debugging changes in recipes to_compile and other dependent tasks.
> I'm also cleaning up sstate to be sure it's not corrupt or filled with
> somehow bad data.

This is bad and shouldn't be happening. Can anyone provide some
examples I can look at?

You shouldn't ever need to run cleansstate in particular. If you have
to, there is another underlying bug that should get fixed.

Cheers,

Richard

Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

[Mikko Rapeli]

On Tue, Mar 10, 2020 at 11:37:55AM +0000, Richard Purdie wrote:
> On Tue, 2020-03-10 at 11:33 +0000, Mikko.Rapeli@bmw.de wrote:
> > On Fri, Feb 28, 2020 at 11:25:59PM +0000, Richard Purdie wrote:
> > > On Fri, 2020-02-28 at 22:51 +0000, Sean McKay wrote:
> > > > This is probably a fairly short question (I hope):
> > > > I’m working on a branch based on zeus. I found a bug in the way that
> > > > one of our recipes was handling something during do_configure (which
> > > > caused an error to pop up in a do_compile task). When I modified the
> > > > do_configure task (do_configure_append, actually) to behave properly
> > > > (which involved changing the actual commands run in that function)
> > > > and reran bitbake -c compile <recipe>, the do_configure task wasn’t
> > > > rerun despite the change to the function’s code.
> > > >  
> > > > Is it safe to assume this means we’ve done something to mess up the
> > > > way our system is processing things? Or is that expected behavior
> > > 
> > > Its not expected behaviour and yes, it sounds like something is messed
> > > up...
> > 
> > This is what I've come to expect with yocto 2.0 jethro, 2.5 sumo and
> > 3.0 zeus.
> > 
> > That's why I always write:
> > 
> > $ bitbake -c clean recipe && bitbake -c cleansstate && bitbake -c compile recipe
> > 
> > when debugging changes in recipes to_compile and other dependent tasks.
> > I'm also cleaning up sstate to be sure it's not corrupt or filled with
> > somehow bad data.
> 
> This is bad and shouldn't be happening. Can anyone provide some
> examples I can look at?
> 
> You shouldn't ever need to run cleansstate in particular. If you have
> to, there is another underlying bug that should get fixed.

Sadly custom bbclasses and BSP layers have this effect on my builds and I
can't fully trust local incremental builds. Hence I clean sstate cache
often.

-Mikko

Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

[rpjday]

Quoting Mikko Rapeli <mikko.rapeli@bmw.de>:

> On Tue, Mar 10, 2020 at 11:37:55AM +0000, Richard Purdie wrote:
>> On Tue, 2020-03-10 at 11:33 +0000, Mikko.Rapeli@bmw.de wrote:
>> > On Fri, Feb 28, 2020 at 11:25:59PM +0000, Richard Purdie wrote:
>> > > On Fri, 2020-02-28 at 22:51 +0000, Sean McKay wrote:
>> > > > This is probably a fairly short question (I hope):
>> > > > I’m working on a branch based on zeus. I found a bug in the way that
>> > > > one of our recipes was handling something during do_configure (which
>> > > > caused an error to pop up in a do_compile task). When I modified the
>> > > > do_configure task (do_configure_append, actually) to behave properly
>> > > > (which involved changing the actual commands run in that function)
>> > > > and reran bitbake -c compile <recipe>, the do_configure task wasn’t
>> > > > rerun despite the change to the function’s code.
>> > > >
>> > > > Is it safe to assume this means we’ve done something to mess up the
>> > > > way our system is processing things? Or is that expected behavior
>> > >
>> > > Its not expected behaviour and yes, it sounds like something is messed
>> > > up...
>> >
>> > This is what I've come to expect with yocto 2.0 jethro, 2.5 sumo and
>> > 3.0 zeus.
>> >
>> > That's why I always write:
>> >
>> > $ bitbake -c clean recipe && bitbake -c cleansstate && bitbake -c  
>> compile recipe
>> >
>> > when debugging changes in recipes to_compile and other dependent tasks.
>> > I'm also cleaning up sstate to be sure it's not corrupt or filled with
>> > somehow bad data.
>>
>> This is bad and shouldn't be happening. Can anyone provide some
>> examples I can look at?
>>
>> You shouldn't ever need to run cleansstate in particular. If you have
>> to, there is another underlying bug that should get fixed.
>
> Sadly custom bbclasses and BSP layers have this effect on my builds and I
> can't fully trust local incremental builds. Hence I clean sstate cache
> often.

Have you verified with "bitbake -e" that the configure task was really
appended to WRT the build? That's the first thing I would check.

rday

Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

[Sean McKay]

[-- Attachment #1: Type: text/plain, Size: 2645 bytes --]

I actually went on vacation right after I sent that message. (Thank you for your incredibly quick reply, by the way!)
Once I get back late next week, I'll try to see how reproducible it is in our environment and if so, what the minimum steps are to do so.

Thanks!
-Sean

~Sean
Sent from my phone. Please forgive terseness and typos!

________________________________
From: Mikko.Rapeli@bmw.de <Mikko.Rapeli@bmw.de>
Sent: Wednesday, March 11, 2020 12:45:21 PM
To: richard.purdie@linuxfoundation.org <richard.purdie@linuxfoundation.org>
Cc: McKay, Sean <sean.mckay@hpe.com>; yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org>
Subject: Re: [yocto] Should changing a task in the .bb file cause the task to be rerun?

On Tue, Mar 10, 2020 at 11:37:55AM +0000, Richard Purdie wrote:
> On Tue, 2020-03-10 at 11:33 +0000, Mikko.Rapeli@bmw.de wrote:
> > On Fri, Feb 28, 2020 at 11:25:59PM +0000, Richard Purdie wrote:
> > > On Fri, 2020-02-28 at 22:51 +0000, Sean McKay wrote:
> > > > This is probably a fairly short question (I hope):
> > > > I’m working on a branch based on zeus. I found a bug in the way that
> > > > one of our recipes was handling something during do_configure (which
> > > > caused an error to pop up in a do_compile task). When I modified the
> > > > do_configure task (do_configure_append, actually) to behave properly
> > > > (which involved changing the actual commands run in that function)
> > > > and reran bitbake -c compile <recipe>, the do_configure task wasn’t
> > > > rerun despite the change to the function’s code.
> > > >
> > > > Is it safe to assume this means we’ve done something to mess up the
> > > > way our system is processing things? Or is that expected behavior
> > >
> > > Its not expected behaviour and yes, it sounds like something is messed
> > > up...
> >
> > This is what I've come to expect with yocto 2.0 jethro, 2.5 sumo and
> > 3.0 zeus.
> >
> > That's why I always write:
> >
> > $ bitbake -c clean recipe && bitbake -c cleansstate && bitbake -c compile recipe
> >
> > when debugging changes in recipes to_compile and other dependent tasks.
> > I'm also cleaning up sstate to be sure it's not corrupt or filled with
> > somehow bad data.
>
> This is bad and shouldn't be happening. Can anyone provide some
> examples I can look at?
>
> You shouldn't ever need to run cleansstate in particular. If you have
> to, there is another underlying bug that should get fixed.

Sadly custom bbclasses and BSP layers have this effect on my builds and I
can't fully trust local incremental builds. Hence I clean sstate cache
often.

-Mikko

[-- Attachment #2: Type: text/html, Size: 4589 bytes --]