* [PATCH 11/16] lib: enable lzma decompression support for SPL build
@ 2020-01-08 3:01 Weijie Gao
2020-01-08 7:22 ` Heinrich Schuchardt
0 siblings, 1 reply; 6+ messages in thread
From: Weijie Gao @ 2020-01-08 3:01 UTC (permalink / raw)
To: u-boot
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao <weijie.gao@mediatek.com>
---
lib/Kconfig | 5 +++++
lib/Makefile | 1 +
2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig
index 965cf7bc03..2f38e37802 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -418,6 +418,11 @@ config SPL_LZ4
fast compression and decompression speed. It belongs to the LZ77
family of byte-oriented compression schemes.
+config SPL_LZMA
+ bool "Enable LZMA decompression support for SPL build"
+ help
+ This enables support for LZMA compression altorithm for SPL boot.
+
config SPL_LZO
bool "Enable LZO decompression support in SPL"
help
diff --git a/lib/Makefile b/lib/Makefile
index 1fb650cd90..8e51eec01c 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/
obj-$(CONFIG_$(SPL_)ZSTD) += zstd/
obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o
obj-$(CONFIG_$(SPL_)LZO) += lzo/
+obj-$(CONFIG_$(SPL_)LZMA) += lzma/
obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
--
2.17.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build
2020-01-08 3:01 [PATCH 11/16] lib: enable lzma decompression support for SPL build Weijie Gao
@ 2020-01-08 7:22 ` Heinrich Schuchardt
2020-01-09 8:10 ` Weijie Gao
0 siblings, 1 reply; 6+ messages in thread
From: Heinrich Schuchardt @ 2020-01-08 7:22 UTC (permalink / raw)
To: u-boot
On 1/8/20 4:01 AM, Weijie Gao wrote:
> This patch enables LZMA decompression support for SPL build
>
> Signed-off-by: Weijie Gao <weijie.gao@mediatek.com>
> ---
> lib/Kconfig | 5 +++++
> lib/Makefile | 1 +
> 2 files changed, 6 insertions(+)
>
> diff --git a/lib/Kconfig b/lib/Kconfig
> index 965cf7bc03..2f38e37802 100644
> --- a/lib/Kconfig
> +++ b/lib/Kconfig
> @@ -418,6 +418,11 @@ config SPL_LZ4
> fast compression and decompression speed. It belongs to the LZ77
> family of byte-oriented compression schemes.
>
> +config SPL_LZMA
> + bool "Enable LZMA decompression support for SPL build"
> + help
> + This enables support for LZMA compression altorithm for SPL boot.
%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression?
Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
> +
> config SPL_LZO
> bool "Enable LZO decompression support in SPL"
> help
> diff --git a/lib/Makefile b/lib/Makefile
> index 1fb650cd90..8e51eec01c 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/
> obj-$(CONFIG_$(SPL_)ZSTD) += zstd/
> obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o
> obj-$(CONFIG_$(SPL_)LZO) += lzo/
> +obj-$(CONFIG_$(SPL_)LZMA) += lzma/
> obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
>
> obj-$(CONFIG_LIBAVB) += libavb/
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build
2020-01-08 7:22 ` Heinrich Schuchardt
@ 2020-01-09 8:10 ` Weijie Gao
2020-01-09 17:45 ` Tom Rini
0 siblings, 1 reply; 6+ messages in thread
From: Weijie Gao @ 2020-01-09 8:10 UTC (permalink / raw)
To: u-boot
On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
>
> On 1/8/20 4:01 AM, Weijie Gao wrote:
> > This patch enables LZMA decompression support for SPL build
> >
> > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com>
> > ---
> > lib/Kconfig | 5 +++++
> > lib/Makefile | 1 +
> > 2 files changed, 6 insertions(+)
> >
> > diff --git a/lib/Kconfig b/lib/Kconfig
> > index 965cf7bc03..2f38e37802 100644
> > --- a/lib/Kconfig
> > +++ b/lib/Kconfig
> > @@ -418,6 +418,11 @@ config SPL_LZ4
> > fast compression and decompression speed. It belongs to the LZ77
> > family of byte-oriented compression schemes.
> >
> > +config SPL_LZMA
> > + bool "Enable LZMA decompression support for SPL build"
> > + help
> > + This enables support for LZMA compression altorithm for SPL boot.
>
> %s/altorithm/algorithm/g
>
> Why do we need yet another algorithm for compression?
>
> Doesn't Zstandard provide a much faster decompression?
> Cf. https://sites.google.com/site/powturbo/home/web-compression
>
> Best regards
>
> Heinrich
>
> > +
> > config SPL_LZO
> > bool "Enable LZO decompression support in SPL"
> > help
> > diff --git a/lib/Makefile b/lib/Makefile
> > index 1fb650cd90..8e51eec01c 100644
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/
> > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/
> > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o
> > obj-$(CONFIG_$(SPL_)LZO) += lzo/
> > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/
> > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
> >
> > obj-$(CONFIG_LIBAVB) += libavb/
> >
Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the
size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices.
Refer to:
https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO
Although they will take longer time for compression and decompression,
considering that the u-boot payload is usually small (<= 1MiB), there
will not be significant effect on decompression time.
Best Regards,
Weijie
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build
2020-01-09 8:10 ` Weijie Gao
@ 2020-01-09 17:45 ` Tom Rini
2020-01-10 8:21 ` Weijie Gao
0 siblings, 1 reply; 6+ messages in thread
From: Tom Rini @ 2020-01-09 17:45 UTC (permalink / raw)
To: u-boot
On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote:
> On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
> >
> > On 1/8/20 4:01 AM, Weijie Gao wrote:
> > > This patch enables LZMA decompression support for SPL build
> > >
> > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com>
> > > ---
> > > lib/Kconfig | 5 +++++
> > > lib/Makefile | 1 +
> > > 2 files changed, 6 insertions(+)
> > >
> > > diff --git a/lib/Kconfig b/lib/Kconfig
> > > index 965cf7bc03..2f38e37802 100644
> > > --- a/lib/Kconfig
> > > +++ b/lib/Kconfig
> > > @@ -418,6 +418,11 @@ config SPL_LZ4
> > > fast compression and decompression speed. It belongs to the LZ77
> > > family of byte-oriented compression schemes.
> > >
> > > +config SPL_LZMA
> > > + bool "Enable LZMA decompression support for SPL build"
> > > + help
> > > + This enables support for LZMA compression altorithm for SPL boot.
> >
> > %s/altorithm/algorithm/g
> >
> > Why do we need yet another algorithm for compression?
> >
> > Doesn't Zstandard provide a much faster decompression?
> > Cf. https://sites.google.com/site/powturbo/home/web-compression
> >
> > Best regards
> >
> > Heinrich
> >
> > > +
> > > config SPL_LZO
> > > bool "Enable LZO decompression support in SPL"
> > > help
> > > diff --git a/lib/Makefile b/lib/Makefile
> > > index 1fb650cd90..8e51eec01c 100644
> > > --- a/lib/Makefile
> > > +++ b/lib/Makefile
> > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/
> > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/
> > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o
> > > obj-$(CONFIG_$(SPL_)LZO) += lzo/
> > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/
> > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
> > >
> > > obj-$(CONFIG_LIBAVB) += libavb/
> > >
>
> Hi Heinrich,
>
> I need an algorithm with the maximum compression ratio to reduce the
> size of the u-boot payload as much as possible.
>
> I believe currently lzma and xz are the best choices.
> Refer to:
> https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO
>
> Although they will take longer time for compression and decompression,
> considering that the u-boot payload is usually small (<= 1MiB), there
> will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and
report back? Thanks!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200109/744b4b1c/attachment.sig>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build
2020-01-09 17:45 ` Tom Rini
@ 2020-01-10 8:21 ` Weijie Gao
2020-01-10 14:17 ` Tom Rini
0 siblings, 1 reply; 6+ messages in thread
From: Weijie Gao @ 2020-01-10 8:21 UTC (permalink / raw)
To: u-boot
On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote:
> On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote:
> > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
> > >
> > > On 1/8/20 4:01 AM, Weijie Gao wrote:
> > > > This patch enables LZMA decompression support for SPL build
> > > >
> > > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com>
> > > > ---
> > > > lib/Kconfig | 5 +++++
> > > > lib/Makefile | 1 +
> > > > 2 files changed, 6 insertions(+)
> > > >
> > > > diff --git a/lib/Kconfig b/lib/Kconfig
> > > > index 965cf7bc03..2f38e37802 100644
> > > > --- a/lib/Kconfig
> > > > +++ b/lib/Kconfig
> > > > @@ -418,6 +418,11 @@ config SPL_LZ4
> > > > fast compression and decompression speed. It belongs to the LZ77
> > > > family of byte-oriented compression schemes.
> > > >
> > > > +config SPL_LZMA
> > > > + bool "Enable LZMA decompression support for SPL build"
> > > > + help
> > > > + This enables support for LZMA compression altorithm for SPL boot.
> > >
> > > %s/altorithm/algorithm/g
> > >
> > > Why do we need yet another algorithm for compression?
> > >
> > > Doesn't Zstandard provide a much faster decompression?
> > > Cf. https://sites.google.com/site/powturbo/home/web-compression
> > >
> > > Best regards
> > >
> > > Heinrich
> > >
> > > > +
> > > > config SPL_LZO
> > > > bool "Enable LZO decompression support in SPL"
> > > > help
> > > > diff --git a/lib/Makefile b/lib/Makefile
> > > > index 1fb650cd90..8e51eec01c 100644
> > > > --- a/lib/Makefile
> > > > +++ b/lib/Makefile
> > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/
> > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/
> > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o
> > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/
> > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/
> > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
> > > >
> > > > obj-$(CONFIG_LIBAVB) += libavb/
> > > >
> >
> > Hi Heinrich,
> >
> > I need an algorithm with the maximum compression ratio to reduce the
> > size of the u-boot payload as much as possible.
> >
> > I believe currently lzma and xz are the best choices.
> > Refer to:
> > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO
> >
> > Although they will take longer time for compression and decompression,
> > considering that the u-boot payload is usually small (<= 1MiB), there
> > will not be significant effect on decompression time.
>
> Where does zstd fall in the list however? Can you please try that and
> report back? Thanks!
>
My purpose is to reduce the size of u-boot binary, and my plan is to
compress the payload and the SPL decompress it at runtime.
I need smallest size here, not fastest decompression speed.
So there are two points must be taken into consideration:
the size of the decompressor (i.e. u-boot-spl.bin) and the size of the
compressed payload (u-boot.bin.gz/bz2/...).
Below is my test results, including all compression algorithms supported
by uImage, plus zstd:
Platform: mips - mt7628
Uncompressed size of u-boot.bin: 302736 (295KiB)
Commands for compression:
gzip -c -f -k -9
bzip2 -c -f -k -9
lzma -c -z -k -9
lzop -c -f -9
lz4 -f -9
zstd --ultra -f
Results:
algo. | spl.bin size | compressed size | % of uncomp | total size
------------------------------------------------------------------------
gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB)
------------------------------------------------------------------------
bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB)
------------------------------------------------------------------------
lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB)
------------------------------------------------------------------------
lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB)
------------------------------------------------------------------------
lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB)
------------------------------------------------------------------------
zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB)
As you can see lzma has the best compression ratio. The is the reason I
chose lzma here.
Best Regards
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build
2020-01-10 8:21 ` Weijie Gao
@ 2020-01-10 14:17 ` Tom Rini
0 siblings, 0 replies; 6+ messages in thread
From: Tom Rini @ 2020-01-10 14:17 UTC (permalink / raw)
To: u-boot
On Fri, Jan 10, 2020 at 04:21:35PM +0800, Weijie Gao wrote:
> On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote:
> > On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote:
> > > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
> > > >
> > > > On 1/8/20 4:01 AM, Weijie Gao wrote:
> > > > > This patch enables LZMA decompression support for SPL build
> > > > >
> > > > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com>
> > > > > ---
> > > > > lib/Kconfig | 5 +++++
> > > > > lib/Makefile | 1 +
> > > > > 2 files changed, 6 insertions(+)
> > > > >
> > > > > diff --git a/lib/Kconfig b/lib/Kconfig
> > > > > index 965cf7bc03..2f38e37802 100644
> > > > > --- a/lib/Kconfig
> > > > > +++ b/lib/Kconfig
> > > > > @@ -418,6 +418,11 @@ config SPL_LZ4
> > > > > fast compression and decompression speed. It belongs to the LZ77
> > > > > family of byte-oriented compression schemes.
> > > > >
> > > > > +config SPL_LZMA
> > > > > + bool "Enable LZMA decompression support for SPL build"
> > > > > + help
> > > > > + This enables support for LZMA compression altorithm for SPL boot.
> > > >
> > > > %s/altorithm/algorithm/g
> > > >
> > > > Why do we need yet another algorithm for compression?
> > > >
> > > > Doesn't Zstandard provide a much faster decompression?
> > > > Cf. https://sites.google.com/site/powturbo/home/web-compression
> > > >
> > > > Best regards
> > > >
> > > > Heinrich
> > > >
> > > > > +
> > > > > config SPL_LZO
> > > > > bool "Enable LZO decompression support in SPL"
> > > > > help
> > > > > diff --git a/lib/Makefile b/lib/Makefile
> > > > > index 1fb650cd90..8e51eec01c 100644
> > > > > --- a/lib/Makefile
> > > > > +++ b/lib/Makefile
> > > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/
> > > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/
> > > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o
> > > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/
> > > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/
> > > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
> > > > >
> > > > > obj-$(CONFIG_LIBAVB) += libavb/
> > > > >
> > >
> > > Hi Heinrich,
> > >
> > > I need an algorithm with the maximum compression ratio to reduce the
> > > size of the u-boot payload as much as possible.
> > >
> > > I believe currently lzma and xz are the best choices.
> > > Refer to:
> > > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO
> > >
> > > Although they will take longer time for compression and decompression,
> > > considering that the u-boot payload is usually small (<= 1MiB), there
> > > will not be significant effect on decompression time.
> >
> > Where does zstd fall in the list however? Can you please try that and
> > report back? Thanks!
> >
>
> My purpose is to reduce the size of u-boot binary, and my plan is to
> compress the payload and the SPL decompress it at runtime.
> I need smallest size here, not fastest decompression speed.
>
> So there are two points must be taken into consideration:
> the size of the decompressor (i.e. u-boot-spl.bin) and the size of the
> compressed payload (u-boot.bin.gz/bz2/...).
>
> Below is my test results, including all compression algorithms supported
> by uImage, plus zstd:
>
> Platform: mips - mt7628
>
> Uncompressed size of u-boot.bin: 302736 (295KiB)
>
> Commands for compression:
> gzip -c -f -k -9
> bzip2 -c -f -k -9
> lzma -c -z -k -9
> lzop -c -f -9
> lz4 -f -9
> zstd --ultra -f
>
> Results:
> algo. | spl.bin size | compressed size | % of uncomp | total size
> ------------------------------------------------------------------------
> gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB)
> ------------------------------------------------------------------------
> bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB)
> ------------------------------------------------------------------------
> lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB)
> ------------------------------------------------------------------------
> lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB)
> ------------------------------------------------------------------------
> lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB)
> ------------------------------------------------------------------------
> zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB)
>
> As you can see lzma has the best compression ratio. The is the reason I
> chose lzma here.
Thanks, that's exactly the kind of data I was looking for.
Reviewed-by: Tom Rini <trini@konsulko.com>
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200110/e5894b33/attachment.sig>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-01-10 14:17 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-08 3:01 [PATCH 11/16] lib: enable lzma decompression support for SPL build Weijie Gao
2020-01-08 7:22 ` Heinrich Schuchardt
2020-01-09 8:10 ` Weijie Gao
2020-01-09 17:45 ` Tom Rini
2020-01-10 8:21 ` Weijie Gao
2020-01-10 14:17 ` Tom Rini
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.