Thank you all for the feedback! I like the idea of switching completely to zstd. I will update the patch accordingly and resubmit. Kind regards, Henry ________________________________ From: Joshua Watt Sent: 01 October 2021 14:44 To: Richard Purdie Cc: Peter Kjellerstedt ; Henry Kleynhans ; poky@lists.yoctoproject.org ; Michael van der Westhuizen Subject: Re: [poky] [PATCH] sstate: Add ZStandard compressor support On Fri, Oct 1, 2021 at 8:16 AM Richard Purdie wrote: > > On Fri, 2021-10-01 at 13:13 +0000, Peter Kjellerstedt wrote: > > > -----Original Message----- > > > From: poky@lists.yoctoproject.org On Behalf > > > Of Henry Kleynhans via lists.yoctoproject.org > > > Sent: den 1 oktober 2021 12:11 > > > To: poky@lists.yoctoproject.org > > > Cc: hkleynhans@fb.com; rmikey@fb.com > > > Subject: [poky] [PATCH] sstate: Add ZStandard compressor support > > > > > > This patch adds support to optionally use the Zstandard compressor for > > > ssate cache files. > > > > > > Zstandard compression provides a significant improvement in > > > decompression speed as well as improvement in compression speed and disk > > > usage over the 'tgz' format in use. Furthermore, its configurable > > > compression level offers a trade-off between time spent compressing > > > sstate cache files and disk space used by those files. The reduced disk > > > usage also contributes to saving network traffic for those sharing their > > > sstate cache with others. > > > > > > Zstandard should therefore be a good choice when: > > > * disk space is at a premium > > > * network speed / resources are limited > > > * the CI server can sstate packages can be created at high compression > > > * less CPU on the build server should be used for sstate decompression > > > > > > Signed-off-by: Henry Kleynhans > > > --- > > > meta/classes/sstate.bbclass | 49 +++++++++++++++++++++++------- > > > scripts/sstate-cache-management.sh | 40 ++++++++++++------------ > > > 2 files changed, 58 insertions(+), 31 deletions(-) > > > > > > diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass > > > index 92a73114bb..a73d631679 100644 > > > --- a/meta/classes/sstate.bbclass > > > +++ b/meta/classes/sstate.bbclass > > > @@ -1,17 +1,30 @@ > > > SSTATE_VERSION = "3" > > > > > > +SSTATE_USE_ZSTD ?= "0" > > > +SSTATE_ZSTD_CLEVEL ?= "3" > > > +SSTATE_ZSTD_NTHREADS ?= "0" > > > > Do we really need to make this configurable? Can't we just decide > > to use zstd and be done with it? > > I have to admit I was also wondering that. We started requiring zstd on the host > system so it may just be better to switch over? Since zstd is required and a clear winner in compression/decompression time, I think requiring it and simplifying the sstate code to only use zstd would be fine. > > Cheers, > > Richard > > > > > >