From: Dennis Zhou <dennis@kernel.org> To: David Sterba <dsterba@suse.com>, Chris Mason <clm@fb.com>, Josef Bacik <josef@toxicpanda.com>, Omar Sandoval <osandov@osandov.com> Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, Dennis Zhou <dennis@kernel.org> Subject: [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Date: Mon, 9 Dec 2019 11:46:06 -0800 Message-ID: <2f96529bfb5a4a3b8e8f9012c1ca2a16f6f000d0.1575919746.git.dennis@kernel.org> (raw) In-Reply-To: <cover.1575919745.git.dennis@kernel.org> In-Reply-To: <cover.1575919745.git.dennis@kernel.org> Currently, there is no way for the free space cache to recover from being serviced by purely bitmaps because the extent threshold is set to 0 in recalculate_thresholds() when we surpass the metadata allowance. This adds a recovery mechanism by keeping large extents out of the bitmaps and increases the metadata upper bound to 64KB. The recovery mechanism bypasses this upper bound, thus making it a soft upper bound. But, with the bypass being 1MB or greater, it shouldn't add unbounded overhead. Signed-off-by: Dennis Zhou <dennis@kernel.org> Reviewed-by: Josef Bacik <josef@toxicpanda.com> --- fs/btrfs/free-space-cache.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 612726a91e98..6a313946a8da 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -24,7 +24,8 @@ #include "discard.h" #define BITS_PER_BITMAP (PAGE_SIZE * 8UL) -#define MAX_CACHE_BYTES_PER_GIG SZ_32K +#define MAX_CACHE_BYTES_PER_GIG SZ_64K +#define FORCE_EXTENT_THRESHOLD SZ_1M struct btrfs_trim_range { u64 start; @@ -1695,26 +1696,17 @@ static void recalculate_thresholds(struct btrfs_free_space_ctl *ctl) ASSERT(ctl->total_bitmaps <= max_bitmaps); /* - * The goal is to keep the total amount of memory used per 1gb of space - * at or below 32k, so we need to adjust how much memory we allow to be - * used by extent based free space tracking + * We are trying to keep the total amount of memory used per 1gb of + * space to be MAX_CACHE_BYTES_PER_GIG. However, with a reclamation + * mechanism of pulling extents >= FORCE_EXTENT_THRESHOLD out of + * bitmaps, we may end up using more memory than this. */ if (size < SZ_1G) max_bytes = MAX_CACHE_BYTES_PER_GIG; else max_bytes = MAX_CACHE_BYTES_PER_GIG * div_u64(size, SZ_1G); - /* - * we want to account for 1 more bitmap than what we have so we can make - * sure we don't go over our overall goal of MAX_CACHE_BYTES_PER_GIG as - * we add more bitmaps. - */ - bitmap_bytes = (ctl->total_bitmaps + 1) * ctl->unit; - - if (bitmap_bytes >= max_bytes) { - ctl->extents_thresh = 0; - return; - } + bitmap_bytes = ctl->total_bitmaps * ctl->unit; /* * we want the extent entry threshold to always be at most 1/2 the max @@ -2100,6 +2092,10 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl, forced = true; #endif + /* This is a way to reclaim large regions from the bitmaps. */ + if (!forced && info->bytes >= FORCE_EXTENT_THRESHOLD) + return false; + /* * If we are below the extents threshold then we can add this as an * extent, and don't have to deal with the bitmap -- 2.17.1
next prev parent reply index Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-09 19:45 [PATCH v5 00/22] btrfs: async discard support Dennis Zhou 2019-12-09 19:45 ` [PATCH 01/22] bitmap: genericize percpu bitmap region iterators Dennis Zhou 2019-12-09 19:45 ` [PATCH 02/22] btrfs: rename DISCARD opt to DISCARD_SYNC Dennis Zhou 2019-12-09 19:45 ` [PATCH 03/22] btrfs: keep track of which extents have been discarded Dennis Zhou 2019-12-09 19:45 ` [PATCH 04/22] btrfs: keep track of cleanliness of the bitmap Dennis Zhou 2019-12-09 19:45 ` [PATCH 05/22] btrfs: add the beginning of async discard, discard workqueue Dennis Zhou 2019-12-09 19:45 ` [PATCH 06/22] btrfs: handle empty block_group removal Dennis Zhou 2019-12-09 19:45 ` [PATCH 07/22] btrfs: discard one region at a time in async discard Dennis Zhou 2019-12-09 19:45 ` [PATCH 08/22] btrfs: add removal calls for sysfs debug/ Dennis Zhou 2019-12-09 19:45 ` [PATCH 09/22] btrfs: make UUID/debug have its own kobject Dennis Zhou 2019-12-09 19:45 ` [PATCH 10/22] btrfs: add discard sysfs directory Dennis Zhou 2019-12-09 19:45 ` [PATCH 11/22] btrfs: track discardable extents for async discard Dennis Zhou 2019-12-09 19:45 ` [PATCH 12/22] btrfs: keep track of discardable_bytes Dennis Zhou 2019-12-09 19:45 ` [PATCH 13/22] btrfs: calculate discard delay based on number of extents Dennis Zhou 2019-12-09 19:45 ` [PATCH 14/22] btrfs: add bps discard rate limit Dennis Zhou 2019-12-09 19:46 ` [PATCH 15/22] btrfs: limit max discard size for async discard Dennis Zhou 2019-12-09 19:46 ` [PATCH 16/22] btrfs: make max async discard size tunable Dennis Zhou 2019-12-09 19:46 ` [PATCH 17/22] btrfs: have multiple discard lists Dennis Zhou 2019-12-09 19:46 ` [PATCH 18/22] btrfs: only keep track of data extents for async discard Dennis Zhou 2019-12-09 19:46 ` [PATCH 19/22] btrfs: keep track of discard reuse stats Dennis Zhou 2019-12-09 19:46 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou 2019-12-09 19:46 ` Dennis Zhou [this message] 2019-12-09 19:46 ` [PATCH 22/22] btrfs: make smaller extents more likely to go into bitmaps Dennis Zhou 2019-12-10 14:04 ` [PATCH v5 00/22] btrfs: async discard support David Sterba -- strict thread matches above, loose matches on Subject: below -- 2019-12-14 0:22 [PATCH v6 " Dennis Zhou 2019-12-14 0:22 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou 2019-11-25 19:46 [PATCH v4 00/22] btrfs: async discard support Dennis Zhou 2019-11-25 19:47 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou 2019-11-20 21:50 [PATCH v3 00/22] btrfs: async discard support Dennis Zhou 2019-11-20 21:51 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou 2019-10-23 22:52 [PATCH v2 00/22] btrfs: async discard support Dennis Zhou 2019-10-23 22:53 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=2f96529bfb5a4a3b8e8f9012c1ca2a16f6f000d0.1575919746.git.dennis@kernel.org \ --to=dennis@kernel.org \ --cc=clm@fb.com \ --cc=dsterba@suse.com \ --cc=josef@toxicpanda.com \ --cc=kernel-team@fb.com \ --cc=linux-btrfs@vger.kernel.org \ --cc=osandov@osandov.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-BTRFS Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \ linux-btrfs@vger.kernel.org public-inbox-index linux-btrfs Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs AGPL code for this site: git clone https://public-inbox.org/public-inbox.git