All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dennis Zhou <dennis@kernel.org>
To: David Sterba <dsterba@suse.com>,
	Josef Bacik <josef@toxicpanda.com>, Chris Mason <clm@fb.com>,
	Omar Sandoval <osandov@osandov.com>,
	Nick Terrell <terrelln@fb.com>,
	Nikolay Borisov <nborisov@suse.com>
Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org,
	linux-kernel@vger.kernel.org, Dennis Zhou <dennis@kernel.org>
Subject: [PATCH 11/12] btrfs: make zstd memory requirements monotonic
Date: Mon,  4 Feb 2019 15:20:07 -0500	[thread overview]
Message-ID: <20190204202008.51652-12-dennis@kernel.org> (raw)
In-Reply-To: <20190204202008.51652-1-dennis@kernel.org>

It is possible based on the level configurations that a higher level
workspace uses less memory than a lower level workspace. In order to
reuse workspaces, this must be made a monotonic relationship. This
precomputes the required memory for each level and enforces the
monotonicity between level and memory required. This is also done
in upstream zstd in [1].

[1] https://github.com/facebook/zstd/commit/a68b76afefec6876f8e8a538155109a5aeac0143

Signed-off-by: Dennis Zhou <dennis@kernel.org>
Cc: Nick Terrell <terrelln@fb.com>
---
 fs/btrfs/zstd.c | 38 +++++++++++++++++++++++++++++++++-----
 1 file changed, 33 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c
index a951d4fe77f7..2231123fedbe 100644
--- a/fs/btrfs/zstd.c
+++ b/fs/btrfs/zstd.c
@@ -20,6 +20,7 @@
 #define ZSTD_BTRFS_MAX_WINDOWLOG 17
 #define ZSTD_BTRFS_MAX_INPUT (1 << ZSTD_BTRFS_MAX_WINDOWLOG)
 #define ZSTD_BTRFS_DEFAULT_LEVEL 3
+#define ZSTD_BTRFS_MAX_LEVEL 15
 
 static ZSTD_parameters zstd_get_btrfs_parameters(unsigned int level,
 						 size_t src_len)
@@ -44,8 +45,39 @@ struct workspace {
 
 static struct workspace_manager wsm;
 
+static size_t zstd_ws_mem_sizes[ZSTD_BTRFS_MAX_LEVEL];
+
+/*
+ * zstd_calc_ws_mem_sizes - calculate monotonic memory bounds
+ *
+ * It is possible based on the level configurations that a higher level
+ * workspace uses less memory than a lower level workspace.  In order to
+ * reuse workspaces, this must be made a monotonic relationship.  This
+ * precomputes the required memory for each level and enforces the
+ * monotonicity between level and memory required.
+ */
+static void zstd_calc_ws_mem_sizes(void)
+{
+	size_t max_size = 0;
+	unsigned int level;
+
+	for (level = 1; level <= ZSTD_BTRFS_MAX_LEVEL; level++) {
+		ZSTD_parameters params =
+			zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT);
+		size_t level_size =
+			max_t(size_t,
+			      ZSTD_CStreamWorkspaceBound(params.cParams),
+			      ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT));
+
+		max_size = max_t(size_t, max_size, level_size);
+		zstd_ws_mem_sizes[level - 1] = max_size;
+	}
+}
+
 static void zstd_init_workspace_manager(void)
 {
+	zstd_calc_ws_mem_sizes();
+
 	btrfs_init_workspace_manager(&wsm, &btrfs_zstd_compress);
 }
 
@@ -80,17 +112,13 @@ static void zstd_free_workspace(struct list_head *ws)
 
 static struct list_head *zstd_alloc_workspace(unsigned int level)
 {
-	ZSTD_parameters params =
-			zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT);
 	struct workspace *workspace;
 
 	workspace = kzalloc(sizeof(*workspace), GFP_KERNEL);
 	if (!workspace)
 		return ERR_PTR(-ENOMEM);
 
-	workspace->size = max_t(size_t,
-			ZSTD_CStreamWorkspaceBound(params.cParams),
-			ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT));
+	workspace->size = zstd_ws_mem_sizes[level - 1];
 	workspace->mem = kvmalloc(workspace->size, GFP_KERNEL);
 	workspace->buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
 	if (!workspace->mem || !workspace->buf)
-- 
2.17.1


  parent reply	other threads:[~2019-02-04 20:21 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-04 20:19 [PATCH v2 00/12] btrfs: add zstd compression level support Dennis Zhou
2019-02-04 20:19 ` [PATCH 01/12] btrfs: add helpers for compression type and level Dennis Zhou
2019-02-04 20:19 ` [PATCH 02/12] btrfs: rename workspaces_list to workspace_manager Dennis Zhou
2019-02-04 20:19 ` [PATCH 03/12] btrfs: manage heuristic workspace as index 0 Dennis Zhou
2019-02-04 20:20 ` [PATCH 04/12] btrfs: unify compression ops with workspace_manager Dennis Zhou
2019-02-04 20:20 ` [PATCH 05/12] btrfs: add helper methods for workspace manager init and cleanup Dennis Zhou
2019-02-04 20:20 ` [PATCH 06/12] btrfs: add compression interface in (get/put)_workspace() Dennis Zhou
2019-02-04 20:20 ` [PATCH 07/12] btrfs: move to fn pointers for get/put workspaces Dennis Zhou
2019-02-04 20:20 ` [PATCH 08/12] btrfs: plumb level through the compression interface Dennis Zhou
2019-02-04 20:20 ` [PATCH 09/12] btrfs: change set_level() to bound the level passed in Dennis Zhou
2019-02-05 19:06   ` David Sterba
2019-02-05 19:32     ` Dennis Zhou
2019-02-05 19:54       ` David Sterba
2019-02-05 19:59         ` Dennis Zhou
2019-02-06 16:48   ` [PATCH v2 " Dennis Zhou
2019-02-04 20:20 ` [PATCH 10/12] btrfs: zstd use the passed through level instead of default Dennis Zhou
2019-02-04 20:20 ` Dennis Zhou [this message]
2019-02-04 20:20 ` [PATCH 12/12] btrfs: add zstd compression level support Dennis Zhou
2019-02-05 19:26   ` [PATCH v2 " Dennis Zhou
2019-02-06 16:47   ` [PATCH v3 " Dennis Zhou
2019-02-05 14:57 ` [PATCH v2 00/12] " David Sterba
2019-02-05 16:03   ` Dennis Zhou
2019-02-05 16:27     ` David Sterba
2019-02-05 16:30       ` Dennis Zhou
2019-02-05 16:51         ` David Sterba
2019-02-05 17:07           ` David Sterba
2019-02-05 18:27             ` David Sterba
2019-02-05 18:30               ` Dennis Zhou
2019-02-05 20:48                 ` Dennis Zhou
2019-02-06 15:15                   ` David Sterba
2019-02-06 16:51                     ` Dennis Zhou
2019-02-07 16:59 ` David Sterba
2019-02-07 17:36   ` Dennis Zhou
2019-02-08 13:22     ` David Sterba

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190204202008.51652-12-dennis@kernel.org \
    --to=dennis@kernel.org \
    --cc=clm@fb.com \
    --cc=dsterba@suse.com \
    --cc=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nborisov@suse.com \
    --cc=osandov@osandov.com \
    --cc=terrelln@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.