Re: [PATCH v2] mkfs.ubifs: Add ZSTD compression

From: Emil Lenngren <emil.lenngren@gmail.com>
To: Richard Weinberger <richard@sigma-star.at>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>,
	linux-mtd@lists.infradead.org,
	David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Subject: Re: [PATCH v2] mkfs.ubifs: Add ZSTD compression
Date: Mon, 10 Jun 2019 19:34:53 +0200	[thread overview]
Message-ID: <CAO1O6sc4RfdrDGiJgXYZaDeZ2e46Rr=PFhqFYYWB9Gqd8XH8NQ@mail.gmail.com> (raw)
In-Reply-To: <1380627689.233779.1559939049026.JavaMail.zimbra@sigma-star.at>

Hi all,

Den fre 7 juni 2019 kl 22:24 skrev Richard Weinberger <richard@sigma-star.at>:
>
> Emil,
>
> ----- Ursprüngliche Mail -----
> > The new mtd-utils with this patch doesn't compile on Ubuntu 18.04 LTS
> > since it uses a slightly older version of zstd (1.3.3) that didn't
> > have the macro ZSTD_CLEVEL_DEFAULT defined, which was introduced in
> > version 1.3.5. Could you maybe consider adding the following lines in
> > compr.c:
> >
> > #ifndef ZSTD_CLEVEL_DEFAULT
> > #define ZSTD_CLEVEL_DEFAULT 3
> > #endif
> >
> > or similar, which make it compile on slightly older distributions?
>
> If I read the docs correctly, we can just pass 0 as compression level,
> which will default to ZSTD_CLEVEL_DEFAULT.
> https://github.com/facebook/zstd/pull/1174/commits/e34c000e44444b9f8bd62e5af0a355ee186eb21f
>
> This should work on all zstd versions.

After some more investigations, although increasing compression level
certainly increases compression time, decompression time does not seem
to be increased by increasing compression level. See
http://www.open-zfs.org/w/images/b/b3/03-OpenZFS_2017_-_ZStandard_in_ZFS.pdf
page 9 for a benchmark. The benchmark even shows this seems to apply
to gz as well...

SquashFS has also added support for zstd and squashfs-tools uses level
15 as the default level (see
https://git.kernel.org/pub/scm/fs/squashfs/squashfs-tools.git/commit/?id=e38956b92f738518c29734399629e7cdb33072d3
at the bottom).

While the kernel compression level should maybe stay at 3, for
mkfs.ubifs where speed doesn't matter that much, a higher level such
as 15 might not be bad after all. So I have two different proposals:
either just set level 15 OR set level 15 and also provide an option
for mkfs.ubifs to override it if one for some reason wants to generate
the image faster.

What do you think?

/Emil

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/