Lifecycle issues in Bitbake

Lifecycle issues in Bitbake

[Richard Purdie]

Looking at and thinking about:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187

We have several issues. Firstly, DATE and TIME are locked in stone when
the base configuration is parsed. This may cause images to overwrite
previous versions for example if DATE and TIME are not re-evaluated. We
could fix this by specifically setting DATE and TIME in bitbake at
BuildStart?

Secondly, there is one cooker log file opened for the duration of the
cooker lifecycle. This is a cooker log, not a log of each build so that
is perhaps not surprising (and is the core problem in the above bug).
There is no overwriting of the log file, it just keeps growing with each
build.

We could trigger knotty to write out build specific log files based on
the BuildStart/Finish events?

Any thoughts on this?

Cheers,

Richard

Re: Lifecycle issues in Bitbake

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Thu, Jan 22, 2015 at 7:57 AM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> Looking at and thinking about:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187
>
> We have several issues. Firstly, DATE and TIME are locked in stone when
> the base configuration is parsed. This may cause images to overwrite
> previous versions for example if DATE and TIME are not re-evaluated. We
> could fix this by specifically setting DATE and TIME in bitbake at
> BuildStart?
>

Seems quite reasonable to me.

Secondly, there is one cooker log file opened for the duration of the
> cooker lifecycle. This is a cooker log, not a log of each build so that
> is perhaps not surprising (and is the core problem in the above bug).
> There is no overwriting of the log file, it just keeps growing with each
> build.
>
> We could trigger knotty to write out build specific log files based on
> the BuildStart/Finish events?
>

This doesn't seem unreasonable, though a build isn't necessarily bound to a
particular UI, is it? Should that go into the server or perhaps the ui
helper at least? Hmm.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2047 bytes --]

Re: Lifecycle issues in Bitbake

[Paul Eggleton]

Hi Richard,

On Thursday 22 January 2015 14:57:19 Richard Purdie wrote:
> Looking at and thinking about:
> 
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187
> 
> We have several issues. Firstly, DATE and TIME are locked in stone when
> the base configuration is parsed. This may cause images to overwrite
> previous versions for example if DATE and TIME are not re-evaluated. We
> could fix this by specifically setting DATE and TIME in bitbake at
> BuildStart?

We could certainly do this. Another mechanism I did think about was to have 
some way of indicating to BitBake that on buildstart the value should be reset 
to the immediate evaluation of an expression; perhaps a varflag. At least that 
way we could handle other such variables, although it's conceivable that 
having built-in DATE and TIME in BitBake wouldn't be the worst thing in the 
world.
 
> Secondly, there is one cooker log file opened for the duration of the
> cooker lifecycle. This is a cooker log, not a log of each build so that
> is perhaps not surprising (and is the core problem in the above bug).
> There is no overwriting of the log file, it just keeps growing with each
> build.
> 
> We could trigger knotty to write out build specific log files based on
> the BuildStart/Finish events?

Yes I think we should do this.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Lifecycle issues in Bitbake

[Alex Damian]

[-- Attachment #1: Type: text/plain, Size: 2955 bytes --]

My 2 cents -

In managed mode, this bug is not a issue for Toaster as we start and stop
the bitbake server separately for each build, for unrelated reasons. This
only appears for the "interactive" model, when toaster logs commands ran
from the CLI.

Details below.

Cheers,
Alex


On Fri, Jan 23, 2015 at 10:33 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> Hi Richard,
>
> On Thursday 22 January 2015 14:57:19 Richard Purdie wrote:
> > Looking at and thinking about:
> >
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=5187
> >
> > We have several issues. Firstly, DATE and TIME are locked in stone when
> > the base configuration is parsed. This may cause images to overwrite
> > previous versions for example if DATE and TIME are not re-evaluated. We
> > could fix this by specifically setting DATE and TIME in bitbake at
> > BuildStart?
>
> We could certainly do this. Another mechanism I did think about was to have
> some way of indicating to BitBake that on buildstart the value should be
> reset
> to the immediate evaluation of an expression; perhaps a varflag. At least
> that
> way we could handle other such variables, although it's conceivable that
> having built-in DATE and TIME in BitBake wouldn't be the worst thing in the
> world.
>

I think this would be the most elegant solution; however I think there is a
catch-22 here, as the build will not start until the datastore is parsed
and evaluated, and we need to parse and evaluate the entire datastore as a
variable as wildly used as DATE and TIME (and maybe others) change at the
start of the build .

I'm not seeing a clean get-away from this ...


>
> > Secondly, there is one cooker log file opened for the duration of the
> > cooker lifecycle. This is a cooker log, not a log of each build so that
> > is perhaps not surprising (and is the core problem in the above bug).
> > There is no overwriting of the log file, it just keeps growing with each
> > build.
> >
> > We could trigger knotty to write out build specific log files based on
> > the BuildStart/Finish events?
>
> Yes I think we should do this.
>

This is not very helpful for Toaster; Toaster relies on the server-side
written cooker log file to have information above what the event system is
exposing, and thus it provides the cookerlog file as an optional
downloadable item for the user's benefit.

If we drop the cooker log we'll have a bit of the gap in the information we
provide to the user.

Multiple builds per cooker is not a problem, since, in managed mode,
Toaster will stop the server after each build for unrelated reasons.


>
> Cheers,
> Paul
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre
> --
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>

[-- Attachment #2: Type: text/html, Size: 4163 bytes --]