From: Dennis Zhou <dennis@kernel.org>
To: David Sterba <dsterba@suse.com>,
Josef Bacik <josef@toxicpanda.com>, Chris Mason <clm@fb.com>,
Omar Sandoval <osandov@osandov.com>,
Nick Terrell <terrelln@fb.com>,
Nikolay Borisov <nborisov@suse.com>
Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org,
linux-kernel@vger.kernel.org, Dennis Zhou <dennis@kernel.org>
Subject: [PATCH 11/12] btrfs: make zstd memory requirements monotonic
Date: Mon, 4 Feb 2019 15:20:07 -0500 [thread overview]
Message-ID: <20190204202008.51652-12-dennis@kernel.org> (raw)
In-Reply-To: <20190204202008.51652-1-dennis@kernel.org>
It is possible based on the level configurations that a higher level
workspace uses less memory than a lower level workspace. In order to
reuse workspaces, this must be made a monotonic relationship. This
precomputes the required memory for each level and enforces the
monotonicity between level and memory required. This is also done
in upstream zstd in [1].
[1] https://github.com/facebook/zstd/commit/a68b76afefec6876f8e8a538155109a5aeac0143
Signed-off-by: Dennis Zhou <dennis@kernel.org>
Cc: Nick Terrell <terrelln@fb.com>
---
fs/btrfs/zstd.c | 38 +++++++++++++++++++++++++++++++++-----
1 file changed, 33 insertions(+), 5 deletions(-)
diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c
index a951d4fe77f7..2231123fedbe 100644
--- a/fs/btrfs/zstd.c
+++ b/fs/btrfs/zstd.c
@@ -20,6 +20,7 @@
#define ZSTD_BTRFS_MAX_WINDOWLOG 17
#define ZSTD_BTRFS_MAX_INPUT (1 << ZSTD_BTRFS_MAX_WINDOWLOG)
#define ZSTD_BTRFS_DEFAULT_LEVEL 3
+#define ZSTD_BTRFS_MAX_LEVEL 15
static ZSTD_parameters zstd_get_btrfs_parameters(unsigned int level,
size_t src_len)
@@ -44,8 +45,39 @@ struct workspace {
static struct workspace_manager wsm;
+static size_t zstd_ws_mem_sizes[ZSTD_BTRFS_MAX_LEVEL];
+
+/*
+ * zstd_calc_ws_mem_sizes - calculate monotonic memory bounds
+ *
+ * It is possible based on the level configurations that a higher level
+ * workspace uses less memory than a lower level workspace. In order to
+ * reuse workspaces, this must be made a monotonic relationship. This
+ * precomputes the required memory for each level and enforces the
+ * monotonicity between level and memory required.
+ */
+static void zstd_calc_ws_mem_sizes(void)
+{
+ size_t max_size = 0;
+ unsigned int level;
+
+ for (level = 1; level <= ZSTD_BTRFS_MAX_LEVEL; level++) {
+ ZSTD_parameters params =
+ zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT);
+ size_t level_size =
+ max_t(size_t,
+ ZSTD_CStreamWorkspaceBound(params.cParams),
+ ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT));
+
+ max_size = max_t(size_t, max_size, level_size);
+ zstd_ws_mem_sizes[level - 1] = max_size;
+ }
+}
+
static void zstd_init_workspace_manager(void)
{
+ zstd_calc_ws_mem_sizes();
+
btrfs_init_workspace_manager(&wsm, &btrfs_zstd_compress);
}
@@ -80,17 +112,13 @@ static void zstd_free_workspace(struct list_head *ws)
static struct list_head *zstd_alloc_workspace(unsigned int level)
{
- ZSTD_parameters params =
- zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT);
struct workspace *workspace;
workspace = kzalloc(sizeof(*workspace), GFP_KERNEL);
if (!workspace)
return ERR_PTR(-ENOMEM);
- workspace->size = max_t(size_t,
- ZSTD_CStreamWorkspaceBound(params.cParams),
- ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT));
+ workspace->size = zstd_ws_mem_sizes[level - 1];
workspace->mem = kvmalloc(workspace->size, GFP_KERNEL);
workspace->buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!workspace->mem || !workspace->buf)
--
2.17.1
next prev parent reply other threads:[~2019-02-04 20:21 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-04 20:19 [PATCH v2 00/12] btrfs: add zstd compression level support Dennis Zhou
2019-02-04 20:19 ` [PATCH 01/12] btrfs: add helpers for compression type and level Dennis Zhou
2019-02-04 20:19 ` [PATCH 02/12] btrfs: rename workspaces_list to workspace_manager Dennis Zhou
2019-02-04 20:19 ` [PATCH 03/12] btrfs: manage heuristic workspace as index 0 Dennis Zhou
2019-02-04 20:20 ` [PATCH 04/12] btrfs: unify compression ops with workspace_manager Dennis Zhou
2019-02-04 20:20 ` [PATCH 05/12] btrfs: add helper methods for workspace manager init and cleanup Dennis Zhou
2019-02-04 20:20 ` [PATCH 06/12] btrfs: add compression interface in (get/put)_workspace() Dennis Zhou
2019-02-04 20:20 ` [PATCH 07/12] btrfs: move to fn pointers for get/put workspaces Dennis Zhou
2019-02-04 20:20 ` [PATCH 08/12] btrfs: plumb level through the compression interface Dennis Zhou
2019-02-04 20:20 ` [PATCH 09/12] btrfs: change set_level() to bound the level passed in Dennis Zhou
2019-02-05 19:06 ` David Sterba
2019-02-05 19:32 ` Dennis Zhou
2019-02-05 19:54 ` David Sterba
2019-02-05 19:59 ` Dennis Zhou
2019-02-06 16:48 ` [PATCH v2 " Dennis Zhou
2019-02-04 20:20 ` [PATCH 10/12] btrfs: zstd use the passed through level instead of default Dennis Zhou
2019-02-04 20:20 ` Dennis Zhou [this message]
2019-02-04 20:20 ` [PATCH 12/12] btrfs: add zstd compression level support Dennis Zhou
2019-02-05 19:26 ` [PATCH v2 " Dennis Zhou
2019-02-06 16:47 ` [PATCH v3 " Dennis Zhou
2019-02-05 14:57 ` [PATCH v2 00/12] " David Sterba
2019-02-05 16:03 ` Dennis Zhou
2019-02-05 16:27 ` David Sterba
2019-02-05 16:30 ` Dennis Zhou
2019-02-05 16:51 ` David Sterba
2019-02-05 17:07 ` David Sterba
2019-02-05 18:27 ` David Sterba
2019-02-05 18:30 ` Dennis Zhou
2019-02-05 20:48 ` Dennis Zhou
2019-02-06 15:15 ` David Sterba
2019-02-06 16:51 ` Dennis Zhou
2019-02-07 16:59 ` David Sterba
2019-02-07 17:36 ` Dennis Zhou
2019-02-08 13:22 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190204202008.51652-12-dennis@kernel.org \
--to=dennis@kernel.org \
--cc=clm@fb.com \
--cc=dsterba@suse.com \
--cc=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nborisov@suse.com \
--cc=osandov@osandov.com \
--cc=terrelln@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).