From: Dennis Zhou <dennis@kernel.org>
To: David Sterba <dsterba@suse.com>, Chris Mason <clm@fb.com>,
Josef Bacik <josef@toxicpanda.com>,
Omar Sandoval <osandov@osandov.com>
Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org,
Dennis Zhou <dennis@kernel.org>
Subject: [PATCH 09/12] btrfs: increase the metadata allowance for the free_space_cache
Date: Thu, 2 Jan 2020 16:26:43 -0500 [thread overview]
Message-ID: <49f2d76f892a24fb6c9cd384f4c7ecc04e63522c.1577999991.git.dennis@kernel.org> (raw)
In-Reply-To: <cover.1577999991.git.dennis@kernel.org>
In-Reply-To: <cover.1577999991.git.dennis@kernel.org>
Currently, there is no way for the free space cache to recover from
being serviced by purely bitmaps because the extent threshold is set to
0 in recalculate_thresholds() when we surpass the metadata allowance.
This adds a recovery mechanism by keeping large extents out of the
bitmaps and increases the metadata upper bound to 64KB. The recovery
mechanism bypasses this upper bound, thus making it a soft upper bound.
But, with the bypass being 1MB or greater, it shouldn't add unbounded
overhead.
Signed-off-by: Dennis Zhou <dennis@kernel.org>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
---
fs/btrfs/free-space-cache.c | 26 +++++++++++---------------
1 file changed, 11 insertions(+), 15 deletions(-)
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 8a4a3b9cd544..665f6eb6c828 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -24,7 +24,8 @@
#include "discard.h"
#define BITS_PER_BITMAP (PAGE_SIZE * 8UL)
-#define MAX_CACHE_BYTES_PER_GIG SZ_32K
+#define MAX_CACHE_BYTES_PER_GIG SZ_64K
+#define FORCE_EXTENT_THRESHOLD SZ_1M
struct btrfs_trim_range {
u64 start;
@@ -1694,26 +1695,17 @@ static void recalculate_thresholds(struct btrfs_free_space_ctl *ctl)
ASSERT(ctl->total_bitmaps <= max_bitmaps);
/*
- * The goal is to keep the total amount of memory used per 1gb of space
- * at or below 32k, so we need to adjust how much memory we allow to be
- * used by extent based free space tracking
+ * We are trying to keep the total amount of memory used per 1gb of
+ * space to be MAX_CACHE_BYTES_PER_GIG. However, with a reclamation
+ * mechanism of pulling extents >= FORCE_EXTENT_THRESHOLD out of
+ * bitmaps, we may end up using more memory than this.
*/
if (size < SZ_1G)
max_bytes = MAX_CACHE_BYTES_PER_GIG;
else
max_bytes = MAX_CACHE_BYTES_PER_GIG * div_u64(size, SZ_1G);
- /*
- * we want to account for 1 more bitmap than what we have so we can make
- * sure we don't go over our overall goal of MAX_CACHE_BYTES_PER_GIG as
- * we add more bitmaps.
- */
- bitmap_bytes = (ctl->total_bitmaps + 1) * ctl->unit;
-
- if (bitmap_bytes >= max_bytes) {
- ctl->extents_thresh = 0;
- return;
- }
+ bitmap_bytes = ctl->total_bitmaps * ctl->unit;
/*
* we want the extent entry threshold to always be at most 1/2 the max
@@ -2099,6 +2091,10 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl,
forced = true;
#endif
+ /* This is a way to reclaim large regions from the bitmaps. */
+ if (!forced && info->bytes >= FORCE_EXTENT_THRESHOLD)
+ return false;
+
/*
* If we are below the extents threshold then we can add this as an
* extent, and don't have to deal with the bitmap
--
2.17.1
next prev parent reply other threads:[~2020-01-02 21:27 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-02 21:26 [PATCH 00/12] btrfs: async discard follow up Dennis Zhou
2020-01-02 21:26 ` [PATCH 01/12] btrfs: calculate discard delay based on number of extents Dennis Zhou
2020-01-03 14:38 ` David Sterba
2020-01-02 21:26 ` [PATCH 02/12] btrfs: add bps discard rate limit for async discard Dennis Zhou
2020-01-03 14:40 ` David Sterba
2020-01-02 21:26 ` [PATCH 03/12] btrfs: limit max discard size " Dennis Zhou
2020-01-03 14:41 ` David Sterba
2020-01-02 21:26 ` [PATCH 04/12] btrfs: make max async discard size tunable Dennis Zhou
2020-01-03 14:44 ` David Sterba
2020-01-02 21:26 ` [PATCH 05/12] btrfs: have multiple discard lists Dennis Zhou
2020-01-02 21:26 ` [PATCH 06/12] btrfs: only keep track of data extents for async discard Dennis Zhou
2020-01-02 21:26 ` [PATCH 07/12] btrfs: keep track of discard reuse stats Dennis Zhou
2020-01-02 21:26 ` [PATCH 08/12] btrfs: add async discard header Dennis Zhou
2020-01-02 21:26 ` Dennis Zhou [this message]
2020-01-02 21:26 ` [PATCH 10/12] btrfs: make smaller extents more likely to go into bitmaps Dennis Zhou
2020-01-02 21:26 ` [PATCH 11/12] btrfs: ensure removal of discardable_* in free_bitmap() Dennis Zhou
2020-01-02 21:26 ` [PATCH 12/12] btrfs: add correction to handle -1 edge case in async discard Dennis Zhou
2020-01-03 14:42 ` David Sterba
2020-01-05 20:35 ` Nikolay Borisov
2020-01-06 13:44 ` David Sterba
2020-01-03 14:51 ` [PATCH 00/12] btrfs: async discard follow up David Sterba
2020-01-03 17:43 ` Dennis Zhou
2020-01-06 15:25 ` David Sterba
2020-01-06 17:14 ` Dennis Zhou
2020-01-06 17:37 ` David Sterba
2020-01-06 16:30 ` David Sterba
2020-01-06 17:28 ` Dennis Zhou
2020-01-06 17:49 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=49f2d76f892a24fb6c9cd384f4c7ecc04e63522c.1577999991.git.dennis@kernel.org \
--to=dennis@kernel.org \
--cc=clm@fb.com \
--cc=dsterba@suse.com \
--cc=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=osandov@osandov.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).