From: Dennis Zhou <dennis@kernel.org> To: David Sterba <dsterba@suse.com>, Chris Mason <clm@fb.com>, Josef Bacik <josef@toxicpanda.com>, Omar Sandoval <osandov@osandov.com> Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, Dennis Zhou <dennis@kernel.org> Subject: [PATCH 20/22] btrfs: add async discard header Date: Wed, 20 Nov 2019 16:51:19 -0500 Message-ID: <723469c79c037e12de0c06e8a1ea36fbc3ee562a.1574282259.git.dennis@kernel.org> (raw) In-Reply-To: <cover.1574282259.git.dennis@kernel.org> In-Reply-To: <cover.1574282259.git.dennis@kernel.org> Give a brief overview for how async discard is implemented. Signed-off-by: Dennis Zhou <dennis@kernel.org> Reviewed-by: Josef Bacik <josef@toxicpanda.com> --- fs/btrfs/discard.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/fs/btrfs/discard.c b/fs/btrfs/discard.c index 83618dd1a5b0..98032b5d341b 100644 --- a/fs/btrfs/discard.c +++ b/fs/btrfs/discard.c @@ -1,4 +1,39 @@ // SPDX-License-Identifier: GPL-2.0 +/* + * This contains the logic to handle async discard. + * + * Async discard manages trimming of free space outside of transaction commit. + * Discarding is done by managing the block_groups on a LRU list based on free + * space recency. Two passes are used to first prioritize discarding extents + * and then allow for trimming in the bitmap the best opportunity to coalesce. + * The block_groups are maintained on multiple lists to allow for multiple + * passes with different discard filter requirements. A delayed work item is + * used to manage discarding with timeout determined by a max of the delay + * incurred by the iops rate limit, byte rate limit, and the timeout of max + * delay of BTRFS_DISCARD_MAX_DELAY. + * + * The first list is special to manage discarding of fully free block groups. + * This is necessary because we issue a final trim for a full free block group + * after forgetting it. When a block group becomes unused, instead of directly + * being added to the unused_bgs list, we add it to this first list. Then + * from there, if it becomes fully discarded, we place it onto the unused_bgs + * list. + * + * The in-memory free space cache serves as the backing state for discard. + * Consequently this means there is no persistence. We opt to load all the + * block groups in as not discarded, so the mount case degenerates to the + * crashing case. + * + * As the free space cache uses bitmaps, there exists a tradeoff between + * ease/efficiency for find_free_extent() and the accuracy of discard state. + * Here we opt to let untrimmed regions merge with everything while only letting + * trimmed regions merge with other trimmed regions. This can cause + * overtrimming, but the coalescing benefit seems to be worth it. Additionally, + * bitmap state is tracked as a whole. If we're able to fully trim a bitmap, + * the trimmed flag is set on the bitmap. Otherwise, if an allocation comes in, + * this resets the state and we will retry trimming the whole bitmap. This is a + * tradeoff between discard state accuracy and the cost of accounting. + */ #include <linux/jiffies.h> #include <linux/kernel.h> -- 2.17.1
next prev parent reply index Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-20 21:50 [PATCH v3 00/22] btrfs: async discard support Dennis Zhou 2019-11-20 21:51 ` [PATCH 01/22] bitmap: genericize percpu bitmap region iterators Dennis Zhou 2019-11-20 21:51 ` [PATCH 02/22] btrfs: rename DISCARD opt to DISCARD_SYNC Dennis Zhou 2019-11-20 21:51 ` [PATCH 03/22] btrfs: keep track of which extents have been discarded Dennis Zhou 2019-11-20 21:51 ` [PATCH 04/22] btrfs: keep track of cleanliness of the bitmap Dennis Zhou 2019-11-23 0:17 ` kbuild test robot 2019-11-25 13:59 ` David Sterba 2019-11-25 19:37 ` Dennis Zhou 2019-11-20 21:51 ` [PATCH 05/22] btrfs: add the beginning of async discard, discard workqueue Dennis Zhou 2019-11-20 21:51 ` [PATCH 06/22] btrfs: handle empty block_group removal Dennis Zhou 2019-11-20 21:51 ` [PATCH 07/22] btrfs: discard one region at a time in async discard Dennis Zhou 2019-11-20 21:51 ` [PATCH 08/22] btrfs: add removal calls for sysfs debug/ Dennis Zhou 2019-11-20 21:51 ` [PATCH 09/22] btrfs: make UUID/debug have its own kobject Dennis Zhou 2019-11-20 21:51 ` [PATCH 10/22] btrfs: add discard sysfs directory Dennis Zhou 2019-11-20 21:51 ` [PATCH 11/22] btrfs: track discardable extents for async discard Dennis Zhou 2019-11-20 21:51 ` [PATCH 12/22] btrfs: keep track of discardable_bytes Dennis Zhou 2019-11-20 21:51 ` [PATCH 13/22] btrfs: calculate discard delay based on number of extents Dennis Zhou 2019-11-20 21:51 ` [PATCH 14/22] btrfs: add bps discard rate limit Dennis Zhou 2019-11-20 21:51 ` [PATCH 15/22] btrfs: limit max discard size for async discard Dennis Zhou 2019-11-20 21:51 ` [PATCH 16/22] btrfs: make max async discard size tunable Dennis Zhou 2019-11-20 21:51 ` [PATCH 17/22] btrfs: have multiple discard lists Dennis Zhou 2019-11-20 21:51 ` [PATCH 18/22] btrfs: only keep track of data extents for async discard Dennis Zhou 2019-11-20 21:51 ` [PATCH 19/22] btrfs: keep track of discard reuse stats Dennis Zhou 2019-11-20 21:51 ` Dennis Zhou [this message] 2019-11-20 21:51 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou 2019-11-20 21:51 ` [PATCH 22/22] btrfs: make smaller extents more likely to go into bitmaps Dennis Zhou -- strict thread matches above, loose matches on Subject: below -- 2019-12-14 0:22 [PATCH v6 00/22] btrfs: async discard support Dennis Zhou 2019-12-14 0:22 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou 2019-12-09 19:45 [PATCH v5 00/22] btrfs: async discard support Dennis Zhou 2019-12-09 19:46 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou 2019-11-25 19:46 [PATCH v4 00/22] btrfs: async discard support Dennis Zhou 2019-11-25 19:47 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou 2019-10-23 22:52 [PATCH v2 00/22] btrfs: async discard support Dennis Zhou 2019-10-23 22:53 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=723469c79c037e12de0c06e8a1ea36fbc3ee562a.1574282259.git.dennis@kernel.org \ --to=dennis@kernel.org \ --cc=clm@fb.com \ --cc=dsterba@suse.com \ --cc=josef@toxicpanda.com \ --cc=kernel-team@fb.com \ --cc=linux-btrfs@vger.kernel.org \ --cc=osandov@osandov.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-BTRFS Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \ linux-btrfs@vger.kernel.org public-inbox-index linux-btrfs Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs AGPL code for this site: git clone https://public-inbox.org/public-inbox.git