From: Josef Bacik <josef@toxicpanda.com>
To: kernel-team@fb.com, linux-btrfs@vger.kernel.org
Subject: [PATCH 13/36] btrfs: add ALLOC_CHUNK_FORCE to the flushing code
Date: Tue, 11 Sep 2018 13:57:44 -0400 [thread overview]
Message-ID: <20180911175807.26181-14-josef@toxicpanda.com> (raw)
In-Reply-To: <20180911175807.26181-1-josef@toxicpanda.com>
With my change to no longer take into account the global reserve for
metadata allocation chunks we have this side-effect for mixed block
group fs'es where we are no longer allocating enough chunks for the
data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE
step to the flushing state machine. This will only get used if we've
already made a full loop through the flushing machinery and tried
committing the transaction. If we have then we can try and force a
chunk allocation since we likely need it to make progress. This
resolves the issues I was seeing with the mixed bg tests in xfstests
with my previous patch.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
fs/btrfs/ctree.h | 3 ++-
fs/btrfs/extent-tree.c | 18 +++++++++++++++++-
include/trace/events/btrfs.h | 1 +
3 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 1a2c3b629af2..29db902511c1 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2730,7 +2730,8 @@ enum btrfs_flush_state {
FLUSH_DELALLOC = 5,
FLUSH_DELALLOC_WAIT = 6,
ALLOC_CHUNK = 7,
- COMMIT_TRANS = 8,
+ ALLOC_CHUNK_FORCE = 8,
+ COMMIT_TRANS = 9,
};
int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index cb065c0c0b9b..13441a293c73 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4914,6 +4914,7 @@ static void flush_space(struct btrfs_fs_info *fs_info,
btrfs_end_transaction(trans);
break;
case ALLOC_CHUNK:
+ case ALLOC_CHUNK_FORCE:
trans = btrfs_join_transaction(root);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
@@ -4921,7 +4922,9 @@ static void flush_space(struct btrfs_fs_info *fs_info,
}
ret = do_chunk_alloc(trans,
btrfs_metadata_alloc_profile(fs_info),
- CHUNK_ALLOC_NO_FORCE);
+ (state == ALLOC_CHUNK) ?
+ CHUNK_ALLOC_NO_FORCE :
+ CHUNK_ALLOC_FORCE);
btrfs_end_transaction(trans);
if (ret > 0 || ret == -ENOSPC)
ret = 0;
@@ -5057,6 +5060,19 @@ static void btrfs_async_reclaim_metadata_space(struct work_struct *work)
commit_cycles--;
}
+ /*
+ * We don't want to force a chunk allocation until we've tried
+ * pretty hard to reclaim space. Think of the case where we
+ * free'd up a bunch of space and so have a lot of pinned space
+ * to reclaim. We would rather use that than possibly create a
+ * underutilized metadata chunk. So if this is our first run
+ * through the flushing state machine skip ALLOC_CHUNK_FORCE and
+ * commit the transaction. If nothing has changed the next go
+ * around then we can force a chunk allocation.
+ */
+ if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
+ flush_state++;
+
if (flush_state > COMMIT_TRANS) {
commit_cycles++;
if (commit_cycles > 2) {
diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h
index 7d205e50b09c..fdb23181b5b7 100644
--- a/include/trace/events/btrfs.h
+++ b/include/trace/events/btrfs.h
@@ -1051,6 +1051,7 @@ TRACE_EVENT(btrfs_trigger_flush,
{ FLUSH_DELAYED_REFS_NR, "FLUSH_DELAYED_REFS_NR"}, \
{ FLUSH_DELAYED_REFS, "FLUSH_ELAYED_REFS"}, \
{ ALLOC_CHUNK, "ALLOC_CHUNK"}, \
+ { ALLOC_CHUNK_FORCE, "ALLOC_CHUNK_FORCE"}, \
{ COMMIT_TRANS, "COMMIT_TRANS"})
TRACE_EVENT(btrfs_flush_space,
--
2.14.3
next prev parent reply other threads:[~2018-09-11 22:59 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-11 17:57 [PATCH 00/35][v2] My current patch queue Josef Bacik
2018-09-11 17:57 ` [PATCH 01/36] btrfs: add btrfs_delete_ref_head helper Josef Bacik
2018-09-11 22:52 ` Omar Sandoval
2018-09-11 17:57 ` [PATCH 02/36] btrfs: add cleanup_ref_head_accounting helper Josef Bacik
2018-09-11 17:57 ` [PATCH 03/36] btrfs: cleanup extent_op handling Josef Bacik
2018-09-11 23:24 ` Omar Sandoval
2018-09-11 17:57 ` [PATCH 04/36] btrfs: only track ref_heads in delayed_ref_updates Josef Bacik
2018-09-11 17:57 ` [PATCH 05/36] btrfs: only count ref heads run in __btrfs_run_delayed_refs Josef Bacik
2018-09-11 23:07 ` Omar Sandoval
2018-09-12 17:46 ` David Sterba
2018-09-11 17:57 ` [PATCH 06/36] btrfs: introduce delayed_refs_rsv Josef Bacik
2018-09-11 17:57 ` [PATCH 07/36] btrfs: check if free bgs for commit Josef Bacik
2018-09-11 23:09 ` Omar Sandoval
2018-09-11 17:57 ` [PATCH 08/36] btrfs: dump block_rsv whe dumping space info Josef Bacik
2018-09-11 23:11 ` Omar Sandoval
2018-09-12 17:24 ` David Sterba
2018-09-11 17:57 ` [PATCH 09/36] btrfs: release metadata before running delayed refs Josef Bacik
2018-09-11 17:57 ` [PATCH 10/36] btrfs: protect space cache inode alloc with nofs Josef Bacik
2018-09-11 17:57 ` [PATCH 11/36] btrfs: fix truncate throttling Josef Bacik
2018-09-11 17:57 ` [PATCH 12/36] btrfs: don't use global rsv for chunk allocation Josef Bacik
2018-09-11 17:57 ` Josef Bacik [this message]
2018-09-11 17:57 ` [PATCH 14/36] btrfs: reset max_extent_size properly Josef Bacik
2018-09-18 23:54 ` Omar Sandoval
2018-09-11 17:57 ` [PATCH 15/36] btrfs: don't enospc all tickets on flush failure Josef Bacik
2018-09-11 17:57 ` [PATCH 16/36] btrfs: run delayed iputs before committing Josef Bacik
2018-09-18 23:59 ` Omar Sandoval
2018-09-11 17:57 ` [PATCH 17/36] btrfs: loop in inode_rsv_refill Josef Bacik
2018-09-19 0:17 ` Omar Sandoval
2018-09-11 17:57 ` [PATCH 18/36] btrfs: move the dio_sem higher up the callchain Josef Bacik
2018-09-11 17:57 ` [PATCH 19/36] btrfs: set max_extent_size properly Josef Bacik
2018-09-11 17:57 ` [PATCH 20/36] btrfs: don't use ctl->free_space for max_extent_size Josef Bacik
2018-09-11 17:57 ` [PATCH 21/36] btrfs: reset max_extent_size on clear in a bitmap Josef Bacik
2018-09-11 17:57 ` [PATCH 22/36] btrfs: only run delayed refs if we're committing Josef Bacik
2018-09-11 17:57 ` [PATCH 23/36] btrfs: make sure we create all new bgs Josef Bacik
2018-09-11 17:57 ` [PATCH 24/36] btrfs: assert on non-empty delayed iputs Josef Bacik
2018-09-11 17:57 ` [PATCH 25/36] btrfs: pass delayed_refs_root to btrfs_delayed_ref_lock Josef Bacik
2018-09-11 17:57 ` [PATCH 26/36] btrfs: make btrfs_destroy_delayed_refs use btrfs_delayed_ref_lock Josef Bacik
2018-09-11 17:57 ` [PATCH 27/36] btrfs: make btrfs_destroy_delayed_refs use btrfs_delete_ref_head Josef Bacik
2018-09-11 17:57 ` [PATCH 28/36] btrfs: handle delayed ref head accounting cleanup in abort Josef Bacik
2018-09-11 17:58 ` [PATCH 29/36] btrfs: call btrfs_create_pending_block_groups unconditionally Josef Bacik
2018-09-11 17:58 ` [PATCH 30/36] btrfs: just delete pending bgs if we are aborted Josef Bacik
2018-09-11 17:58 ` [PATCH 31/36] btrfs: cleanup pending bgs on transaction abort Josef Bacik
2018-09-11 17:58 ` [PATCH 32/36] btrfs: clear delayed_refs_rsv for dirty bg cleanup Josef Bacik
2018-09-19 0:21 ` Omar Sandoval
2018-09-11 17:58 ` [PATCH 33/36] btrfs: only free reserved extent if we didn't insert it Josef Bacik
2018-09-19 0:20 ` Omar Sandoval
2018-09-11 17:58 ` [PATCH 34/36] btrfs: fix insert_reserved error handling Josef Bacik
2018-09-11 17:58 ` [PATCH 35/36] btrfs: wait on ordered extents on abort cleanup Josef Bacik
2018-09-11 17:58 ` [PATCH 36/36] MAINTAINERS: update my email address for btrfs Josef Bacik
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180911175807.26181-14-josef@toxicpanda.com \
--to=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).