All of lore.kernel.org
 help / color / mirror / Atom feed
From: Josef Bacik <josef@toxicpanda.com>
To: linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: [PATCH 10/13] btrfs: handle uncontrolled delayed ref generation
Date: Fri, 13 Mar 2020 17:23:27 -0400	[thread overview]
Message-ID: <20200313212330.149024-11-josef@toxicpanda.com> (raw)
In-Reply-To: <20200313212330.149024-1-josef@toxicpanda.com>

Some operations can generate way too many delayed refs, resulting in the
async flusher being unable to keep up.  To deal with this keep track of
how often we're needing to throttle the trans handles, and if it's too
much increase how many delayed refs they need to wait on each iteration.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/delayed-ref.h |  3 +++
 fs/btrfs/transaction.c | 21 +++++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h
index 16cf0af91464..03590a13f86e 100644
--- a/fs/btrfs/delayed-ref.h
+++ b/fs/btrfs/delayed-ref.h
@@ -157,6 +157,9 @@ struct btrfs_delayed_ref_root {
 	atomic_t entries_run;
 	wait_queue_head_t wait;
 
+	atomic_t mult;
+	time64_t last_adjustment;
+
 	/* total number of head nodes in tree */
 	unsigned long num_heads;
 
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index ac77a2b805fa..6f74f9699560 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -308,6 +308,7 @@ static noinline int join_transaction(struct btrfs_fs_info *fs_info,
 	cur_trans->delayed_refs.dirty_extent_root = RB_ROOT;
 	atomic_set(&cur_trans->delayed_refs.num_entries, 0);
 	atomic_set(&cur_trans->delayed_refs.entries_run, 0);
+	atomic_set(&cur_trans->delayed_refs.mult, 1);
 	init_waitqueue_head(&cur_trans->delayed_refs.wait);
 
 	/*
@@ -902,6 +903,17 @@ btrfs_throttle_for_delayed_refs(struct btrfs_fs_info *fs_info,
 {
 	unsigned long threshold = max(refs, 1UL) +
 		atomic_read(&delayed_refs->entries_run);
+	time64_t start = ktime_get_seconds();
+
+	spin_lock(&delayed_refs->lock);
+	if (delayed_refs->last_adjustment - start >= 1) {
+		if (delayed_refs->last_adjustment)
+			atomic_inc(&delayed_refs->mult);
+		delayed_refs->last_adjustment = start;
+	}
+	spin_unlock(&delayed_refs->lock);
+	refs *= atomic_read(&delayed_refs->mult);
+
 	wait_event_interruptible(delayed_refs->wait,
 		 (atomic_read(&delayed_refs->entries_run) >= threshold) ||
 		 !btrfs_should_throttle_delayed_refs(fs_info, delayed_refs,
@@ -973,6 +985,15 @@ static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
 		err = -EIO;
 	}
 
+	if (!throttle_delayed_refs && atomic_read(&cur_trans->delayed_refs.mult) > 1) {
+		time64_t start = ktime_get_seconds();
+		spin_lock(&cur_trans->delayed_refs.lock);
+		if ((start - cur_trans->delayed_refs.last_adjustment) >= 1) {
+			atomic_dec(&cur_trans->delayed_refs.mult);
+			cur_trans->delayed_refs.last_adjustment = start;
+		}
+		spin_unlock(&cur_trans->delayed_refs.lock);
+	}
 	if (run_async && !work_busy(&info->async_delayed_ref_work))
 		queue_work(system_unbound_wq,
 			   &info->async_delayed_ref_work);
-- 
2.24.1


  parent reply	other threads:[~2020-03-13 21:23 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-13 21:23 [PATCH 00/13] Throttle delayed refs based on time Josef Bacik
2020-03-13 21:23 ` [PATCH 01/13] btrfs: use a stable rolling avg for delayed refs avg Josef Bacik
2020-03-13 21:23 ` [PATCH 02/13] btrfs: change btrfs_should_throttle_delayed_refs to a bool Josef Bacik
2020-03-13 21:23 ` [PATCH 03/13] btrfs: make btrfs_should_throttle_delayed_refs only check run time Josef Bacik
2020-03-13 21:23 ` [PATCH 04/13] btrfs: make should_end_transaction check time to run delayed refs Josef Bacik
2020-03-13 21:23 ` [PATCH 05/13] btrfs: squash should_end_transaction Josef Bacik
2020-03-13 21:23 ` [PATCH 06/13] btrfs: add a mode for delayed ref time based throttling Josef Bacik
2020-03-13 21:23 ` [PATCH 07/13] btrfs: kick off async delayed ref flushing if we are over time budget Josef Bacik
2020-04-09 13:11   ` Nikolay Borisov
2020-04-09 13:26   ` Nikolay Borisov
2020-03-13 21:23 ` [PATCH 08/13] btrfs: adjust the arguments for btrfs_should_throttle_delayed_refs Josef Bacik
2020-03-13 21:23 ` [PATCH 09/13] btrfs: throttle delayed refs based on time Josef Bacik
2020-03-13 21:23 ` Josef Bacik [this message]
2020-03-13 21:23 ` [PATCH 11/13] btrfs: check delayed refs while relocating Josef Bacik
2020-03-13 21:23 ` [PATCH 12/13] btrfs: throttle truncate for delayed ref generation Josef Bacik
2020-03-13 21:23 ` [PATCH 13/13] btrfs: throttle snapshot delete on delayed refs Josef Bacik

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200313212330.149024-11-josef@toxicpanda.com \
    --to=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.