Cache timings with various changes

Cache timings with various changes

[Richard Purdie]

I've talked about changing the data we cache in various forums. These
are some actual numbers from a local system (88 core):

Patch                   Full reparse time    Cache load time    Cache size (compressed)     Cache size (uncompressed)
-----                   -----------------    ---------------    -----------------------     -------------------------
master baseline         5.732s               2.261s             N/A                         24M
frozenset changes       4.326s               2.289s             N/A                         24M
pickle in one           4.509s               2.177s             N/A                         24M
compress with zstd      6.040s               2.204s             3.4M                        24M
add hash data to cache  9.693s               4.327s             17M                         83M

It seems the frozenset improvements I have speed things up nicely and
look like a good change. The compression does slow things down, as does
pickling the cache in one go.

The patches are in poky-contrib rpurdie/t222. I'll post the frozenset
one, the others look like they need more thought.

Cheers,

Richard

RE: [bitbake-devel] Cache timings with various changes

[Peter Kjellerstedt]

> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 23 november 2022 01:07
> To: bitbake-devel <bitbake-devel@lists.openembedded.org>
> Cc: Joshua Watt <JPEWhacker@gmail.com>
> Subject: [bitbake-devel] Cache timings with various changes
> 
> I've talked about changing the data we cache in various forums. These
> are some actual numbers from a local system (88 core):
> 
> Patch                   Full reparse time    Cache load time    Cache size (compressed)     Cache size (uncompressed)
> -----                   -----------------    ---------------    -----------------------     -------------------------
> master baseline         5.732s               2.261s             N/A                         24M
> frozenset changes       4.326s               2.289s             N/A                         24M
> pickle in one           4.509s               2.177s             N/A                         24M
> compress with zstd      6.040s               2.204s             3.4M                        24M
> add hash data to cache  9.693s               4.327s             17M                         83M
> 
> It seems the frozenset improvements I have speed things up nicely and
> look like a good change. The compression does slow things down, as does
> pickling the cache in one go.
> 
> The patches are in poky-contrib rpurdie/t222. I'll post the frozenset
> one, the others look like they need more thought.
> 
> Cheers,
> 
> Richard

Not sure how you measured the times, so I just looked at the progress bars 
for "Loading cache" and "Parsing recipes". With our layers (4717 recipes) 
and my computer with 8 cores (16 threads), this is what I got:

* Without rpurdie/t222 applied, it takes 0-2 s to load the cache, it takes 
  43-46 s to parse the recipes and the cache is 185 MB.
* After applying up to "cache: Switch to compressing the cache file with 
  zstd", the cache load and recipe parse times remains the same, but the 
  size of the cache decreases to 17 MB
* With all of rpurdie/t222 applied it takes 2-3 to load the cache, 51-53 s 
  to parse the recipes and the cache is 55 MB.

//Peter

Re: [bitbake-devel] Cache timings with various changes

[Richard Purdie]

On Wed, 2022-11-23 at 04:33 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 23 november 2022 01:07
> > To: bitbake-devel <bitbake-devel@lists.openembedded.org>
> > Cc: Joshua Watt <JPEWhacker@gmail.com>
> > Subject: [bitbake-devel] Cache timings with various changes
> > 
> > I've talked about changing the data we cache in various forums. These
> > are some actual numbers from a local system (88 core):
> > 
> > Patch                   Full reparse time    Cache load time    Cache size (compressed)     Cache size (uncompressed)
> > -----                   -----------------    ---------------    -----------------------     -------------------------
> > master baseline         5.732s               2.261s             N/A                         24M
> > frozenset changes       4.326s               2.289s             N/A                         24M
> > pickle in one           4.509s               2.177s             N/A                         24M
> > compress with zstd      6.040s               2.204s             3.4M                        24M
> > add hash data to cache  9.693s               4.327s             17M                         83M
> > 
> > It seems the frozenset improvements I have speed things up nicely and
> > look like a good change. The compression does slow things down, as does
> > pickling the cache in one go.
> > 
> > The patches are in poky-contrib rpurdie/t222. I'll post the frozenset
> > one, the others look like they need more thought.
> > 
> > Cheers,
> > 
> > Richard
> 
> Not sure how you measured the times, so I just looked at the progress bars 
> for "Loading cache" and "Parsing recipes". 

The numbers are from:

https://git.yoctoproject.org/poky-contrib/commit/?h=rpurdie/t222&id=3f3c3f5df43097ee4539308e6f06d838bbd69f40

i.e. "time bitbake -p" with a hot/cold cache. In that sense they're the
real world numbers a user sees.

> With our layers (4717 recipes) 
> and my computer with 8 cores (16 threads), this is what I got:
> 
> * Without rpurdie/t222 applied, it takes 0-2 s to load the cache, it takes 
>   43-46 s to parse the recipes and the cache is 185 MB.
> * After applying up to "cache: Switch to compressing the cache file with 
>   zstd", the cache load and recipe parse times remains the same, but the 
>   size of the cache decreases to 17 MB
> * With all of rpurdie/t222 applied it takes 2-3 to load the cache, 51-53 s 
>   to parse the recipes and the cache is 55 MB.

That is useful data, thanks! It seems to agree with my estimate that
the patch slows down main parsing by a few seconds (probably something
we could live with) and doubles the hot cache load time (not so good).

Cheers,

Richard

Re: [bitbake-devel] Cache timings with various changes

[Richard Purdie]

On Wed, 2022-11-23 at 00:07 +0000, Richard Purdie via
lists.openembedded.org wrote:
> I've talked about changing the data we cache in various forums. These
> are some actual numbers from a local system (88 core):
> 
> Patch                   Full reparse time    Cache load time    Cache size (compressed)     Cache size (uncompressed)
> -----                   -----------------    ---------------    -----------------------     -------------------------
> master baseline         5.732s               2.261s             N/A                         24M
> frozenset changes       4.326s               2.289s             N/A                         24M
> pickle in one           4.509s               2.177s             N/A                         24M
> compress with zstd      6.040s               2.204s             3.4M                        24M

This line in the data is flawed as I changed the compression algorithm
for codeparser at the same time and therefore the 6s number includes
rebuilding the codeparser cache which none of the others do.

I'll continue to try and work through some of the pros/cons of the
changes.

Cheers,

Richard