Timing a recipe

Timing a recipe

[rustyhowell]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

Is there a way to automatically record build time of a recipe without modifying the recipe itself?  I have a recipe that is a monster, it has many git URIs and produces many packages that are coupled. It should be broken up but company deadlines have kept us from taking the time to do this. I was talking to my boss about this. He said we really needs some concrete data about how much this monster recipe is costing us before green-lighting the massive effort to split up the recipe.  I agree with him, so I would like to know how much time is spent repeatedly building this recipe.  I figure I could do this by adding do_fetch_prepend() and writing the time to a file, and then do_install_append() and writing the time to the file. So after a week or so, I would have many start/end data points to discuss.

However, I did not want to modify the recipe itself. I was hoping to augment the recipe via local.conf. Another bbappend in a new layer would work I guess, but I was hoping there was something simpler. Thanks in advance.

[-- Attachment #2: Type: text/html, Size: 1074 bytes --]

Re: [yocto] Timing a recipe

[Konrad Weihmann]

buildstats [1] maybe? That can be inherited globally and does give some 
neat results and insights

[1] 
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/buildstats.bbclass

On 15.02.21 21:08, rustyhowell@gmail.com wrote:
> Is there a way to automatically record build time of a recipe without 
> modifying the recipe itself?  I have a recipe that is a monster, it has 
> many git URIs and produces many packages that are coupled. It should be 
> broken up but company deadlines have kept us from taking the time to do 
> this. I was talking to my boss about this. He said we really needs some 
> concrete data about how much this monster recipe is costing us before 
> green-lighting the massive effort to split up the recipe.  I agree with 
> him, so I would like to know how much time is spent repeatedly building 
> this recipe.  I figure I could do this by adding do_fetch_prepend() and 
> writing the time to a file, and then do_install_append() and writing the 
> time to the file. So after a week or so, I would have many start/end 
> data points to discuss.
> 
> However, I did not want to modify the recipe itself. I was hoping to 
> augment the recipe via local.conf. Another bbappend in a new layer would 
> work I guess, but I was hoping there was something simpler. Thanks in 
> advance.
> 
> 
> 
> 
> 

Re: [yocto] Timing a recipe

[Yocto]

[-- Attachment #1: Type: text/plain, Size: 1491 bytes --]


On 2/16/21 3:12 AM, Konrad Weihmann wrote:
> buildstats [1] maybe? That can be inherited globally and does give 
> some neat results and insights
>
> [1] 
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/buildstats.bbclass
>
how about "time bitbake recipe"  :)


> On 15.02.21 21:08, rustyhowell@gmail.com wrote:
>> Is there a way to automatically record build time of a recipe without 
>> modifying the recipe itself?  I have a recipe that is a monster, it 
>> has many git URIs and produces many packages that are coupled. It 
>> should be broken up but company deadlines have kept us from taking 
>> the time to do this. I was talking to my boss about this. He said we 
>> really needs some concrete data about how much this monster recipe is 
>> costing us before green-lighting the massive effort to split up the 
>> recipe.  I agree with him, so I would like to know how much time is 
>> spent repeatedly building this recipe.  I figure I could do this by 
>> adding do_fetch_prepend() and writing the time to a file, and then 
>> do_install_append() and writing the time to the file. So after a week 
>> or so, I would have many start/end data points to discuss.
>>
>> However, I did not want to modify the recipe itself. I was hoping to 
>> augment the recipe via local.conf. Another bbappend in a new layer 
>> would work I guess, but I was hoping there was something simpler. 
>> Thanks in advance.
>>
>>
>>
>>
>>
>
> 
>

[-- Attachment #2: Type: text/html, Size: 2610 bytes --]

Re: Timing a recipe

[rustyhowell]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

"time bitbake recipe" is perfect for manual things. But I wanted to also measure the recipe times when building the entire image.  I ended up creating  a bbappend with new pre/post tasks for the main tasks (fetch, unpack, configure, compile, install, package).   The pre task drops a timestamp file and the post task reads the file, calculates the elapsed time and logs it to a file.  It's a bit clunky but it gives the information I want.  Thanks for the help.

[-- Attachment #2: Type: text/html, Size: 493 bytes --]

Re: [yocto] Timing a recipe

[Khem Raj]

On Tue, Feb 16, 2021 at 11:43 AM <rustyhowell@gmail.com> wrote:
>
> "time bitbake recipe" is perfect for manual things. But I wanted to also measure the recipe times when building the entire image.  I ended up creating  a bbappend with new pre/post tasks for the main tasks (fetch, unpack, configure, compile, install, package).   The pre task drops a timestamp file and the post task reads the file, calculates the elapsed time and logs it to a file.  It's a bit clunky but it gives the information I want.  Thanks for the help.

you could also look into using buildstats bbclass
https://wiki.yoctoproject.org/wiki/TipsAndTricks/InvestigatingBuildTime

> 
>

Re: [yocto] Timing a recipe

[Ross Burton]

As said, buildstats is *exactly* what you want.  There's a forked
pybootchart in oe-core that can visualise the data too.

Ross

On Mon, 15 Feb 2021 at 20:08, <rustyhowell@gmail.com> wrote:
>
> Is there a way to automatically record build time of a recipe without modifying the recipe itself?  I have a recipe that is a monster, it has many git URIs and produces many packages that are coupled. It should be broken up but company deadlines have kept us from taking the time to do this. I was talking to my boss about this. He said we really needs some concrete data about how much this monster recipe is costing us before green-lighting the massive effort to split up the recipe.  I agree with him, so I would like to know how much time is spent repeatedly building this recipe.  I figure I could do this by adding do_fetch_prepend() and writing the time to a file, and then do_install_append() and writing the time to the file. So after a week or so, I would have many start/end data points to discuss.
>
> However, I did not want to modify the recipe itself. I was hoping to augment the recipe via local.conf. Another bbappend in a new layer would work I guess, but I was hoping there was something simpler. Thanks in advance.
>
>
> 
>

Re: [yocto] Timing a recipe

[Richard Purdie]

On Tue, 2021-02-16 at 11:43 -0800, rustyhowell@gmail.com wrote:
> "time bitbake recipe" is perfect for manual things. But I wanted to
> also measure the recipe times when building the entire image.  I
> ended up creating  a bbappend with new pre/post tasks for the main
> tasks (fetch, unpack, configure, compile, install, package).   The
> pre task drops a timestamp file and the post task reads the file,
> calculates the elapsed time and logs it to a file.  It's a bit clunky
> but it gives the information I want.  Thanks for the help.

As others have said, please look at the buildstats class and the data
it saves into TMPDIR/buildstats. It should do what you want and we have
tools like pybootchart which can show it visually.

Cheers,

Richard