* [PATCH] kbuild: use smaller dictionary size for zstd module compression
@ 2021-09-10 13:09 Piotr Gorski
2021-09-17 15:07 ` Piotr Górski
0 siblings, 1 reply; 3+ messages in thread
From: Piotr Gorski @ 2021-09-10 13:09 UTC (permalink / raw)
To: linux-kernel
Cc: linux-kbuild, Michal Marek, Masahiro Yamada, Nick Desaulniers,
Piotr Gorski
By default, zstd without parameters uses a dictionary size of 8 MB.
However, most modules are much smaller than that.
Use a dictionary size of 2 MB for module compression, resulting in
slightly higher compression speed while still maintaining a good
compression ratio.
The --zstd=wlog=21 option is equivalent to --lzma2=dict=2MiB used in XZ compression.
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
---
scripts/Makefile.modinst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
index ff9b09e4cfca..c3475e5aca7c 100644
--- a/scripts/Makefile.modinst
+++ b/scripts/Makefile.modinst
@@ -97,7 +97,7 @@ quiet_cmd_gzip = GZIP $@
quiet_cmd_xz = XZ $@
cmd_xz = $(XZ) --lzma2=dict=2MiB -f $<
quiet_cmd_zstd = ZSTD $@
- cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
+ cmd_zstd = $(ZSTD) --zstd=wlog=21 -T0 --rm -f -q $<
$(dst)/%.ko.gz: $(dst)/%.ko FORCE
$(call cmd,gzip)
--
2.33.0.142.ge0a2f5cbc5
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] kbuild: use smaller dictionary size for zstd module compression
2021-09-10 13:09 [PATCH] kbuild: use smaller dictionary size for zstd module compression Piotr Gorski
@ 2021-09-17 15:07 ` Piotr Górski
2021-09-19 11:36 ` Masahiro Yamada
0 siblings, 1 reply; 3+ messages in thread
From: Piotr Górski @ 2021-09-17 15:07 UTC (permalink / raw)
To: Linux Kernel Mailing List
Cc: Linux Kbuild mailing list, Michal Marek, Masahiro Yamada,
Nick Desaulniers
Ping...
pt., 10 wrz 2021 o 15:10 Piotr Gorski <lucjan.lucjanov@gmail.com> napisał(a):
>
> By default, zstd without parameters uses a dictionary size of 8 MB.
> However, most modules are much smaller than that.
> Use a dictionary size of 2 MB for module compression, resulting in
> slightly higher compression speed while still maintaining a good
> compression ratio.
> The --zstd=wlog=21 option is equivalent to --lzma2=dict=2MiB used in XZ compression.
>
> Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
> ---
> scripts/Makefile.modinst | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> index ff9b09e4cfca..c3475e5aca7c 100644
> --- a/scripts/Makefile.modinst
> +++ b/scripts/Makefile.modinst
> @@ -97,7 +97,7 @@ quiet_cmd_gzip = GZIP $@
> quiet_cmd_xz = XZ $@
> cmd_xz = $(XZ) --lzma2=dict=2MiB -f $<
> quiet_cmd_zstd = ZSTD $@
> - cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> + cmd_zstd = $(ZSTD) --zstd=wlog=21 -T0 --rm -f -q $<
>
> $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> $(call cmd,gzip)
> --
> 2.33.0.142.ge0a2f5cbc5
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] kbuild: use smaller dictionary size for zstd module compression
2021-09-17 15:07 ` Piotr Górski
@ 2021-09-19 11:36 ` Masahiro Yamada
0 siblings, 0 replies; 3+ messages in thread
From: Masahiro Yamada @ 2021-09-19 11:36 UTC (permalink / raw)
To: Piotr Górski
Cc: Linux Kernel Mailing List, Linux Kbuild mailing list,
Michal Marek, Nick Desaulniers
On Sat, Sep 18, 2021 at 12:07 AM Piotr Górski <lucjan.lucjanov@gmail.com> wrote:
>
> Ping...
>
> pt., 10 wrz 2021 o 15:10 Piotr Gorski <lucjan.lucjanov@gmail.com> napisał(a):
> >
> > By default, zstd without parameters uses a dictionary size of 8 MB.
> > However, most modules are much smaller than that.
> > Use a dictionary size of 2 MB for module compression, resulting in
> > slightly higher compression speed while still maintaining a good
> > compression ratio.
> > The --zstd=wlog=21 option is equivalent to --lzma2=dict=2MiB used in XZ compression.
What is a benefit of this patch?
It claims "slightly higher compression", but I do not see a difference.
(15.0577 sec --> 15.013 sec)
[1] build allmodconfig
[2] collect *.ko files whose size is less than 2MB
[3] measure the time elapsed for compressing all of them
I did 'perf stat --repeat 10'.
The default zstd:
Performance counter stats for '/tmp/zstd2/test.sh' (10 runs):
19,250.23 msec task-clock # 1.278 CPUs
utilized ( +- 0.49% )
65,424 context-switches # 0.003 M/sec
( +- 0.03% )
5,609 cpu-migrations # 0.291 K/sec
( +- 0.25% )
4,207,419 page-faults # 0.219 M/sec
( +- 0.19% )
54,349,495,038 cycles # 2.823 GHz
( +- 0.15% ) (85.93%)
3,833,225,032 stalled-cycles-frontend # 7.05% frontend
cycles idle ( +- 0.47% ) (86.34%)
20,789,203,221 stalled-cycles-backend # 38.25% backend
cycles idle ( +- 0.31% ) (85.92%)
65,385,071,612 instructions # 1.20 insn per
cycle
# 0.32 stalled
cycles per insn ( +- 0.09% ) (86.54%)
11,033,842,879 branches # 573.180 M/sec
( +- 0.10% ) (86.38%)
288,314,097 branch-misses # 2.61% of all
branches ( +- 0.16% ) (86.33%)
15.0577 +- 0.0728 seconds time elapsed ( +- 0.48% )
zstd --zstd=wlog=21:
Performance counter stats for '/tmp/zstd2/test2.sh' (10 runs):
19,203.08 msec task-clock # 1.279 CPUs
utilized ( +- 0.85% )
65,404 context-switches # 0.003 M/sec
( +- 0.05% )
5,617 cpu-migrations # 0.293 K/sec
( +- 0.15% )
4,214,502 page-faults # 0.219 M/sec
( +- 0.12% )
54,495,218,169 cycles # 2.838 GHz
( +- 0.29% ) (85.55%)
3,867,166,903 stalled-cycles-frontend # 7.10% frontend
cycles idle ( +- 0.26% ) (87.22%)
20,950,577,113 stalled-cycles-backend # 38.44% backend
cycles idle ( +- 0.38% ) (86.51%)
65,500,859,667 instructions # 1.20 insn per
cycle
# 0.32 stalled
cycles per insn ( +- 0.12% ) (86.65%)
11,045,504,296 branches # 575.194 M/sec
( +- 0.12% ) (85.90%)
288,729,169 branch-misses # 2.61% of all
branches ( +- 0.12% ) (85.59%)
15.013 +- 0.129 seconds time elapsed ( +- 0.86% )
> > Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
> > ---
> > scripts/Makefile.modinst | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> > index ff9b09e4cfca..c3475e5aca7c 100644
> > --- a/scripts/Makefile.modinst
> > +++ b/scripts/Makefile.modinst
> > @@ -97,7 +97,7 @@ quiet_cmd_gzip = GZIP $@
> > quiet_cmd_xz = XZ $@
> > cmd_xz = $(XZ) --lzma2=dict=2MiB -f $<
> > quiet_cmd_zstd = ZSTD $@
> > - cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> > + cmd_zstd = $(ZSTD) --zstd=wlog=21 -T0 --rm -f -q $<
> >
> > $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> > $(call cmd,gzip)
> > --
> > 2.33.0.142.ge0a2f5cbc5
> >
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-09-19 11:37 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-10 13:09 [PATCH] kbuild: use smaller dictionary size for zstd module compression Piotr Gorski
2021-09-17 15:07 ` Piotr Górski
2021-09-19 11:36 ` Masahiro Yamada
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).