[PATCH 00/18] Truncate cleanups and preparation work

[PATCH 00/18] Truncate cleanups and preparation work

[Josef Bacik]

Hello,

The first thing I'm implementing with the garbage collection tree is
btrfs_truncate_inode_items() on evicted inodes.  However
btrfs_truncate_inode_items() has a lot of oddities that it's grown over the
years, and requires having a valid btrfs_inode to use.  We don't really want to
have to look up the old inode to do the truncate, we just want to do the tree
operaitons to delete all of the objects and extents.

Enter this patch series, I've cleaned up btrfs_truncate_inode_items(), moved as
much of the inode operations out to the respective callers, and cleaned up the
argument passing and such to make it a little cleaner.

We still have to pass in the inode for the ^NO_HOLES case for the normal
truncate path, but other than that I've stripped it down so that we can pass in
a NULL inode and get all the work done.

This has the nice side-effect of cleaning up a lot of our

if (root == LOG_ROOT)
	// do something else

checks in this helper, and hopefully makes it more straightforward to
understand.  Thanks,

Josef

Josef Bacik (18):
  btrfs: add an inode-item.h
  btrfs: move btrfs_truncate_inode_items to inode-item.c
  btrfs: move extent locking outside of btrfs_truncate_inode_items
  btrfs: remove free space cache inode check in
    btrfs_truncate_inode_items
  btrfs: move btrfs_kill_delayed_inode_items into evict
  btrfs: remove found_extent from btrfs_truncate_inode_items
  btrfs: add btrfs_truncate_control struct
  btrfs: only update i_size in truncate paths that care
  btrfs: only call inode_sub_bytes in truncate paths that care
  btrfs: control extent reference updates with a control flag for
    truncate
  btrfs: use a flag to control when to clear the file extent range
  btrfs: pass the ino via btrfs_truncate_control
  btrfs: add inode to btrfs_truncate_control
  btrfs: convert BUG_ON() in btrfs_truncate_inode_items to ASSERT
  btrfs: convert BUG() for pending_del_nr into an ASSERT
  btrfs: combine extra if statements in btrfs_truncate_inode_items
  btrfs: make should_throttle loop local in btrfs_truncate_inode_items
  btrfs: do not check -EAGAIN when truncating inodes in the log root

 fs/btrfs/ctree.h            |  34 ---
 fs/btrfs/delayed-inode.c    |   1 +
 fs/btrfs/free-space-cache.c |  31 ++-
 fs/btrfs/inode-item.c       | 334 ++++++++++++++++++++++++++
 fs/btrfs/inode-item.h       |  86 +++++++
 fs/btrfs/inode.c            | 452 +++++-------------------------------
 fs/btrfs/relocation.c       |   1 +
 fs/btrfs/tree-log.c         |  15 +-
 8 files changed, 511 insertions(+), 443 deletions(-)
 create mode 100644 fs/btrfs/inode-item.h

-- 
2.26.3

[PATCH 01/18] btrfs: add an inode-item.h

[Josef Bacik]

We have a few helpers in inode-item.c, and I'm going to make a few
changes to how we do truncate in the future, so break out these
definitions into their own header file to trim down ctree.h some and
make it easier to do the work on truncate in the future.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/ctree.h            | 30 ------------------------------
 fs/btrfs/delayed-inode.c    |  1 +
 fs/btrfs/free-space-cache.c |  1 +
 fs/btrfs/inode-item.c       |  1 +
 fs/btrfs/inode-item.h       | 37 +++++++++++++++++++++++++++++++++++++
 fs/btrfs/inode.c            |  1 +
 fs/btrfs/relocation.c       |  1 +
 fs/btrfs/tree-log.c         |  1 +
 8 files changed, 43 insertions(+), 30 deletions(-)
 create mode 100644 fs/btrfs/inode-item.h

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index dfee4b403da1..f33cae82e7dd 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -3126,36 +3126,6 @@ int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,
 			  struct btrfs_root *root, u64 offset);
 int btrfs_find_orphan_item(struct btrfs_root *root, u64 offset);
 
-/* inode-item.c */
-int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
-			   struct btrfs_root *root,
-			   const char *name, int name_len,
-			   u64 inode_objectid, u64 ref_objectid, u64 index);
-int btrfs_del_inode_ref(struct btrfs_trans_handle *trans,
-			   struct btrfs_root *root,
-			   const char *name, int name_len,
-			   u64 inode_objectid, u64 ref_objectid, u64 *index);
-int btrfs_insert_empty_inode(struct btrfs_trans_handle *trans,
-			     struct btrfs_root *root,
-			     struct btrfs_path *path, u64 objectid);
-int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
-		       *root, struct btrfs_path *path,
-		       struct btrfs_key *location, int mod);
-
-struct btrfs_inode_extref *
-btrfs_lookup_inode_extref(struct btrfs_trans_handle *trans,
-			  struct btrfs_root *root,
-			  struct btrfs_path *path,
-			  const char *name, int name_len,
-			  u64 inode_objectid, u64 ref_objectid, int ins_len,
-			  int cow);
-
-struct btrfs_inode_ref *btrfs_find_name_in_backref(struct extent_buffer *leaf,
-						   int slot, const char *name,
-						   int name_len);
-struct btrfs_inode_extref *btrfs_find_name_in_ext_backref(
-		struct extent_buffer *leaf, int slot, u64 ref_objectid,
-		const char *name, int name_len);
 /* file-item.c */
 struct btrfs_dio_private;
 int btrfs_del_csums(struct btrfs_trans_handle *trans,
diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c
index 6f134f2c5e68..748bf6b0d860 100644
--- a/fs/btrfs/delayed-inode.c
+++ b/fs/btrfs/delayed-inode.c
@@ -13,6 +13,7 @@
 #include "ctree.h"
 #include "qgroup.h"
 #include "locking.h"
+#include "inode-item.h"
 
 #define BTRFS_DELAYED_WRITEBACK		512
 #define BTRFS_DELAYED_BACKGROUND	128
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 132209ff2262..55e1be703a39 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -23,6 +23,7 @@
 #include "block-group.h"
 #include "discard.h"
 #include "subpage.h"
+#include "inode-item.h"
 
 #define BITS_PER_BITMAP		(PAGE_SIZE * 8UL)
 #define MAX_CACHE_BYTES_PER_GIG	SZ_64K
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 56755ce9a907..72593a93c43c 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -4,6 +4,7 @@
  */
 
 #include "ctree.h"
+#include "inode-item.h"
 #include "disk-io.h"
 #include "transaction.h"
 #include "print-tree.h"
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
new file mode 100644
index 000000000000..cb4b140e3b7d
--- /dev/null
+++ b/fs/btrfs/inode-item.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef BTRFS_INODE_ITEM_H
+#define BTRFS_INODE_ITEM_H
+
+
+int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
+			   struct btrfs_root *root,
+			   const char *name, int name_len,
+			   u64 inode_objectid, u64 ref_objectid, u64 index);
+int btrfs_del_inode_ref(struct btrfs_trans_handle *trans,
+			   struct btrfs_root *root,
+			   const char *name, int name_len,
+			   u64 inode_objectid, u64 ref_objectid, u64 *index);
+int btrfs_insert_empty_inode(struct btrfs_trans_handle *trans,
+			     struct btrfs_root *root,
+			     struct btrfs_path *path, u64 objectid);
+int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
+		       *root, struct btrfs_path *path,
+		       struct btrfs_key *location, int mod);
+
+struct btrfs_inode_extref *
+btrfs_lookup_inode_extref(struct btrfs_trans_handle *trans,
+			  struct btrfs_root *root,
+			  struct btrfs_path *path,
+			  const char *name, int name_len,
+			  u64 inode_objectid, u64 ref_objectid, int ins_len,
+			  int cow);
+
+struct btrfs_inode_ref *btrfs_find_name_in_backref(struct extent_buffer *leaf,
+						   int slot, const char *name,
+						   int name_len);
+struct btrfs_inode_extref *btrfs_find_name_in_ext_backref(
+		struct extent_buffer *leaf, int slot, u64 ref_objectid,
+		const char *name, int name_len);
+
+#endif /* BTRFS_INODE_ITEM_H */
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 504cf090fc88..6ccdcf76b02f 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -54,6 +54,7 @@
 #include "space-info.h"
 #include "zoned.h"
 #include "subpage.h"
+#include "inode-item.h"
 
 struct btrfs_iget_args {
 	u64 ino;
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 775aff5a2c26..f5465197996d 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -26,6 +26,7 @@
 #include "misc.h"
 #include "subpage.h"
 #include "zoned.h"
+#include "inode-item.h"
 
 /*
  * Relocation overview
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index de79e15a7c6a..c7a7f78708d5 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -20,6 +20,7 @@
 #include "block-group.h"
 #include "space-info.h"
 #include "zoned.h"
+#include "inode-item.h"
 
 /* magic values for the inode_only field in btrfs_log_inode:
  *
-- 
2.26.3

[PATCH 02/18] btrfs: move btrfs_truncate_inode_items to inode-item.c

[Josef Bacik]

This is an inode item related manipulation with a few vfs related
adjustments.  I'm going to remove the vfs related code from this helper
and simplify it a lot, but I want those changes to be easily seen via
git blame, so move this function now and then the simplification work
can be done.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/ctree.h      |   4 -
 fs/btrfs/inode-item.c | 377 ++++++++++++++++++++++++++++++++++++++++
 fs/btrfs/inode-item.h |   9 +
 fs/btrfs/inode.c      | 391 +-----------------------------------------
 4 files changed, 390 insertions(+), 391 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index f33cae82e7dd..02f06ee02e4e 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -3185,10 +3185,6 @@ int btrfs_add_link(struct btrfs_trans_handle *trans,
 int btrfs_delete_subvolume(struct inode *dir, struct dentry *dentry);
 int btrfs_truncate_block(struct btrfs_inode *inode, loff_t from, loff_t len,
 			 int front);
-int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
-			       struct btrfs_root *root,
-			       struct btrfs_inode *inode, u64 new_size,
-			       u32 min_type, u64 *extents_found);
 
 int btrfs_start_delalloc_snapshot(struct btrfs_root *root, bool in_reclaim_context);
 int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, long nr,
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 72593a93c43c..6e4b244ce96c 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -418,3 +418,380 @@ int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
 	}
 	return ret;
 }
+
+/*
+ * Remove inode items from a given root.
+ *
+ * @trans:		A transaction handle.
+ * @root:		The root from which to remove items.
+ * @inode:		The inode whose items we want to remove.
+ * @new_size:		The new i_size for the inode. This is only applicable when
+ *			@min_type is BTRFS_EXTENT_DATA_KEY, must be 0 otherwise.
+ * @min_type:		The minimum key type to remove. All keys with a type
+ *			greater than this value are removed and all keys with
+ *			this type are removed only if their offset is >= @new_size.
+ * @extents_found:	Output parameter that will contain the number of file
+ *			extent items that were removed or adjusted to the new
+ *			inode i_size. The caller is responsible for initializing
+ *			the counter. Also, it can be NULL if the caller does not
+ *			need this counter.
+ *
+ * Remove all keys associated with the inode from the given root that have a key
+ * with a type greater than or equals to @min_type. When @min_type has a value of
+ * BTRFS_EXTENT_DATA_KEY, only remove file extent items that have an offset value
+ * greater than or equals to @new_size. If a file extent item that starts before
+ * @new_size and ends after it is found, its length is adjusted.
+ *
+ * Returns: 0 on success, < 0 on error and NEED_TRUNCATE_BLOCK when @min_type is
+ * BTRFS_EXTENT_DATA_KEY and the caller must truncate the last block.
+ */
+int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
+			       struct btrfs_root *root,
+			       struct btrfs_inode *inode,
+			       u64 new_size, u32 min_type,
+			       u64 *extents_found)
+{
+	struct btrfs_fs_info *fs_info = root->fs_info;
+	struct btrfs_path *path;
+	struct extent_buffer *leaf;
+	struct btrfs_file_extent_item *fi;
+	struct btrfs_key key;
+	struct btrfs_key found_key;
+	u64 extent_start = 0;
+	u64 extent_num_bytes = 0;
+	u64 extent_offset = 0;
+	u64 item_end = 0;
+	u64 last_size = new_size;
+	u32 found_type = (u8)-1;
+	int found_extent;
+	int del_item;
+	int pending_del_nr = 0;
+	int pending_del_slot = 0;
+	int extent_type = -1;
+	int ret;
+	u64 ino = btrfs_ino(inode);
+	u64 bytes_deleted = 0;
+	bool be_nice = false;
+	bool should_throttle = false;
+	const u64 lock_start = ALIGN_DOWN(new_size, fs_info->sectorsize);
+	struct extent_state *cached_state = NULL;
+
+	BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY);
+
+	/*
+	 * For non-free space inodes and non-shareable roots, we want to back
+	 * off from time to time.  This means all inodes in subvolume roots,
+	 * reloc roots, and data reloc roots.
+	 */
+	if (!btrfs_is_free_space_inode(inode) &&
+	    test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
+		be_nice = true;
+
+	path = btrfs_alloc_path();
+	if (!path)
+		return -ENOMEM;
+	path->reada = READA_BACK;
+
+	if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
+		lock_extent_bits(&inode->io_tree, lock_start, (u64)-1,
+				 &cached_state);
+
+		/*
+		 * We want to drop from the next block forward in case this
+		 * new size is not block aligned since we will be keeping the
+		 * last block of the extent just the way it is.
+		 */
+		btrfs_drop_extent_cache(inode, ALIGN(new_size,
+					fs_info->sectorsize),
+					(u64)-1, 0);
+	}
+
+	/*
+	 * This function is also used to drop the items in the log tree before
+	 * we relog the inode, so if root != BTRFS_I(inode)->root, it means
+	 * it is used to drop the logged items. So we shouldn't kill the delayed
+	 * items.
+	 */
+	if (min_type == 0 && root == inode->root)
+		btrfs_kill_delayed_inode_items(inode);
+
+	key.objectid = ino;
+	key.offset = (u64)-1;
+	key.type = (u8)-1;
+
+search_again:
+	/*
+	 * with a 16K leaf size and 128MB extents, you can actually queue
+	 * up a huge file in a single leaf.  Most of the time that
+	 * bytes_deleted is > 0, it will be huge by the time we get here
+	 */
+	if (be_nice && bytes_deleted > SZ_32M &&
+	    btrfs_should_end_transaction(trans)) {
+		ret = -EAGAIN;
+		goto out;
+	}
+
+	ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+	if (ret < 0)
+		goto out;
+
+	if (ret > 0) {
+		ret = 0;
+		/* there are no items in the tree for us to truncate, we're
+		 * done
+		 */
+		if (path->slots[0] == 0)
+			goto out;
+		path->slots[0]--;
+	}
+
+	while (1) {
+		u64 clear_start = 0, clear_len = 0;
+
+		fi = NULL;
+		leaf = path->nodes[0];
+		btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+		found_type = found_key.type;
+
+		if (found_key.objectid != ino)
+			break;
+
+		if (found_type < min_type)
+			break;
+
+		item_end = found_key.offset;
+		if (found_type == BTRFS_EXTENT_DATA_KEY) {
+			fi = btrfs_item_ptr(leaf, path->slots[0],
+					    struct btrfs_file_extent_item);
+			extent_type = btrfs_file_extent_type(leaf, fi);
+			if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
+				item_end +=
+				    btrfs_file_extent_num_bytes(leaf, fi);
+
+				trace_btrfs_truncate_show_fi_regular(
+					inode, leaf, fi, found_key.offset);
+			} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
+				item_end += btrfs_file_extent_ram_bytes(leaf,
+									fi);
+
+				trace_btrfs_truncate_show_fi_inline(
+					inode, leaf, fi, path->slots[0],
+					found_key.offset);
+			}
+			item_end--;
+		}
+		if (found_type > min_type) {
+			del_item = 1;
+		} else {
+			if (item_end < new_size)
+				break;
+			if (found_key.offset >= new_size)
+				del_item = 1;
+			else
+				del_item = 0;
+		}
+		found_extent = 0;
+		/* FIXME, shrink the extent if the ref count is only 1 */
+		if (found_type != BTRFS_EXTENT_DATA_KEY)
+			goto delete;
+
+		if (extents_found != NULL)
+			(*extents_found)++;
+
+		if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
+			u64 num_dec;
+
+			clear_start = found_key.offset;
+			extent_start = btrfs_file_extent_disk_bytenr(leaf, fi);
+			if (!del_item) {
+				u64 orig_num_bytes =
+					btrfs_file_extent_num_bytes(leaf, fi);
+				extent_num_bytes = ALIGN(new_size -
+						found_key.offset,
+						fs_info->sectorsize);
+				clear_start = ALIGN(new_size, fs_info->sectorsize);
+				btrfs_set_file_extent_num_bytes(leaf, fi,
+							 extent_num_bytes);
+				num_dec = (orig_num_bytes -
+					   extent_num_bytes);
+				if (test_bit(BTRFS_ROOT_SHAREABLE,
+					     &root->state) &&
+				    extent_start != 0)
+					inode_sub_bytes(&inode->vfs_inode,
+							num_dec);
+				btrfs_mark_buffer_dirty(leaf);
+			} else {
+				extent_num_bytes =
+					btrfs_file_extent_disk_num_bytes(leaf,
+									 fi);
+				extent_offset = found_key.offset -
+					btrfs_file_extent_offset(leaf, fi);
+
+				/* FIXME blocksize != 4096 */
+				num_dec = btrfs_file_extent_num_bytes(leaf, fi);
+				if (extent_start != 0) {
+					found_extent = 1;
+					if (test_bit(BTRFS_ROOT_SHAREABLE,
+						     &root->state))
+						inode_sub_bytes(&inode->vfs_inode,
+								num_dec);
+				}
+			}
+			clear_len = num_dec;
+		} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
+			/*
+			 * we can't truncate inline items that have had
+			 * special encodings
+			 */
+			if (!del_item &&
+			    btrfs_file_extent_encryption(leaf, fi) == 0 &&
+			    btrfs_file_extent_other_encoding(leaf, fi) == 0 &&
+			    btrfs_file_extent_compression(leaf, fi) == 0) {
+				u32 size = (u32)(new_size - found_key.offset);
+
+				btrfs_set_file_extent_ram_bytes(leaf, fi, size);
+				size = btrfs_file_extent_calc_inline_size(size);
+				btrfs_truncate_item(path, size, 1);
+			} else if (!del_item) {
+				/*
+				 * We have to bail so the last_size is set to
+				 * just before this extent.
+				 */
+				ret = BTRFS_NEED_TRUNCATE_BLOCK;
+				break;
+			} else {
+				/*
+				 * Inline extents are special, we just treat
+				 * them as a full sector worth in the file
+				 * extent tree just for simplicity sake.
+				 */
+				clear_len = fs_info->sectorsize;
+			}
+
+			if (test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
+				inode_sub_bytes(&inode->vfs_inode,
+						item_end + 1 - new_size);
+		}
+delete:
+		/*
+		 * We use btrfs_truncate_inode_items() to clean up log trees for
+		 * multiple fsyncs, and in this case we don't want to clear the
+		 * file extent range because it's just the log.
+		 */
+		if (root == inode->root) {
+			ret = btrfs_inode_clear_file_extent_range(inode,
+						  clear_start, clear_len);
+			if (ret) {
+				btrfs_abort_transaction(trans, ret);
+				break;
+			}
+		}
+
+		if (del_item)
+			last_size = found_key.offset;
+		else
+			last_size = new_size;
+		if (del_item) {
+			if (!pending_del_nr) {
+				/* no pending yet, add ourselves */
+				pending_del_slot = path->slots[0];
+				pending_del_nr = 1;
+			} else if (pending_del_nr &&
+				   path->slots[0] + 1 == pending_del_slot) {
+				/* hop on the pending chunk */
+				pending_del_nr++;
+				pending_del_slot = path->slots[0];
+			} else {
+				BUG();
+			}
+		} else {
+			break;
+		}
+		should_throttle = false;
+
+		if (found_extent &&
+		    root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
+			struct btrfs_ref ref = { 0 };
+
+			bytes_deleted += extent_num_bytes;
+
+			btrfs_init_generic_ref(&ref, BTRFS_DROP_DELAYED_REF,
+					extent_start, extent_num_bytes, 0);
+			btrfs_init_data_ref(&ref, btrfs_header_owner(leaf),
+					ino, extent_offset,
+					root->root_key.objectid, false);
+			ret = btrfs_free_extent(trans, &ref);
+			if (ret) {
+				btrfs_abort_transaction(trans, ret);
+				break;
+			}
+			if (be_nice) {
+				if (btrfs_should_throttle_delayed_refs(trans))
+					should_throttle = true;
+			}
+		}
+
+		if (found_type == BTRFS_INODE_ITEM_KEY)
+			break;
+
+		if (path->slots[0] == 0 ||
+		    path->slots[0] != pending_del_slot ||
+		    should_throttle) {
+			if (pending_del_nr) {
+				ret = btrfs_del_items(trans, root, path,
+						pending_del_slot,
+						pending_del_nr);
+				if (ret) {
+					btrfs_abort_transaction(trans, ret);
+					break;
+				}
+				pending_del_nr = 0;
+			}
+			btrfs_release_path(path);
+
+			/*
+			 * We can generate a lot of delayed refs, so we need to
+			 * throttle every once and a while and make sure we're
+			 * adding enough space to keep up with the work we are
+			 * generating.  Since we hold a transaction here we
+			 * can't flush, and we don't want to FLUSH_LIMIT because
+			 * we could have generated too many delayed refs to
+			 * actually allocate, so just bail if we're short and
+			 * let the normal reservation dance happen higher up.
+			 */
+			if (should_throttle) {
+				ret = btrfs_delayed_refs_rsv_refill(fs_info,
+							BTRFS_RESERVE_NO_FLUSH);
+				if (ret) {
+					ret = -EAGAIN;
+					break;
+				}
+			}
+			goto search_again;
+		} else {
+			path->slots[0]--;
+		}
+	}
+out:
+	if (ret >= 0 && pending_del_nr) {
+		int err;
+
+		err = btrfs_del_items(trans, root, path, pending_del_slot,
+				      pending_del_nr);
+		if (err) {
+			btrfs_abort_transaction(trans, err);
+			ret = err;
+		}
+	}
+	if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
+		ASSERT(last_size >= new_size);
+		if (!ret && last_size > new_size)
+			last_size = new_size;
+		btrfs_inode_safe_disk_i_size_write(inode, last_size);
+		unlock_extent_cached(&inode->io_tree, lock_start, (u64)-1,
+				     &cached_state);
+	}
+
+	btrfs_free_path(path);
+	return ret;
+}
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index cb4b140e3b7d..63e8f45f110f 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -3,7 +3,16 @@
 #ifndef BTRFS_INODE_ITEM_H
 #define BTRFS_INODE_ITEM_H
 
+/*
+ * Return this if we need to call truncate_block for the last bit of the
+ * truncate.
+ */
+#define BTRFS_NEED_TRUNCATE_BLOCK 1
 
+int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
+			       struct btrfs_root *root,
+			       struct btrfs_inode *inode, u64 new_size,
+			       u32 min_type, u64 *extents_found);
 int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
 			   struct btrfs_root *root,
 			   const char *name, int name_len,
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 6ccdcf76b02f..c29e7c87ff27 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -4616,389 +4616,6 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry)
 	return err;
 }
 
-/*
- * Return this if we need to call truncate_block for the last bit of the
- * truncate.
- */
-#define NEED_TRUNCATE_BLOCK 1
-
-/*
- * Remove inode items from a given root.
- *
- * @trans:		A transaction handle.
- * @root:		The root from which to remove items.
- * @inode:		The inode whose items we want to remove.
- * @new_size:		The new i_size for the inode. This is only applicable when
- *			@min_type is BTRFS_EXTENT_DATA_KEY, must be 0 otherwise.
- * @min_type:		The minimum key type to remove. All keys with a type
- *			greater than this value are removed and all keys with
- *			this type are removed only if their offset is >= @new_size.
- * @extents_found:	Output parameter that will contain the number of file
- *			extent items that were removed or adjusted to the new
- *			inode i_size. The caller is responsible for initializing
- *			the counter. Also, it can be NULL if the caller does not
- *			need this counter.
- *
- * Remove all keys associated with the inode from the given root that have a key
- * with a type greater than or equals to @min_type. When @min_type has a value of
- * BTRFS_EXTENT_DATA_KEY, only remove file extent items that have an offset value
- * greater than or equals to @new_size. If a file extent item that starts before
- * @new_size and ends after it is found, its length is adjusted.
- *
- * Returns: 0 on success, < 0 on error and NEED_TRUNCATE_BLOCK when @min_type is
- * BTRFS_EXTENT_DATA_KEY and the caller must truncate the last block.
- */
-int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
-			       struct btrfs_root *root,
-			       struct btrfs_inode *inode,
-			       u64 new_size, u32 min_type,
-			       u64 *extents_found)
-{
-	struct btrfs_fs_info *fs_info = root->fs_info;
-	struct btrfs_path *path;
-	struct extent_buffer *leaf;
-	struct btrfs_file_extent_item *fi;
-	struct btrfs_key key;
-	struct btrfs_key found_key;
-	u64 extent_start = 0;
-	u64 extent_num_bytes = 0;
-	u64 extent_offset = 0;
-	u64 item_end = 0;
-	u64 last_size = new_size;
-	u32 found_type = (u8)-1;
-	int found_extent;
-	int del_item;
-	int pending_del_nr = 0;
-	int pending_del_slot = 0;
-	int extent_type = -1;
-	int ret;
-	u64 ino = btrfs_ino(inode);
-	u64 bytes_deleted = 0;
-	bool be_nice = false;
-	bool should_throttle = false;
-	const u64 lock_start = ALIGN_DOWN(new_size, fs_info->sectorsize);
-	struct extent_state *cached_state = NULL;
-
-	BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY);
-
-	/*
-	 * For non-free space inodes and non-shareable roots, we want to back
-	 * off from time to time.  This means all inodes in subvolume roots,
-	 * reloc roots, and data reloc roots.
-	 */
-	if (!btrfs_is_free_space_inode(inode) &&
-	    test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
-		be_nice = true;
-
-	path = btrfs_alloc_path();
-	if (!path)
-		return -ENOMEM;
-	path->reada = READA_BACK;
-
-	if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
-		lock_extent_bits(&inode->io_tree, lock_start, (u64)-1,
-				 &cached_state);
-
-		/*
-		 * We want to drop from the next block forward in case this
-		 * new size is not block aligned since we will be keeping the
-		 * last block of the extent just the way it is.
-		 */
-		btrfs_drop_extent_cache(inode, ALIGN(new_size,
-					fs_info->sectorsize),
-					(u64)-1, 0);
-	}
-
-	/*
-	 * This function is also used to drop the items in the log tree before
-	 * we relog the inode, so if root != BTRFS_I(inode)->root, it means
-	 * it is used to drop the logged items. So we shouldn't kill the delayed
-	 * items.
-	 */
-	if (min_type == 0 && root == inode->root)
-		btrfs_kill_delayed_inode_items(inode);
-
-	key.objectid = ino;
-	key.offset = (u64)-1;
-	key.type = (u8)-1;
-
-search_again:
-	/*
-	 * with a 16K leaf size and 128MB extents, you can actually queue
-	 * up a huge file in a single leaf.  Most of the time that
-	 * bytes_deleted is > 0, it will be huge by the time we get here
-	 */
-	if (be_nice && bytes_deleted > SZ_32M &&
-	    btrfs_should_end_transaction(trans)) {
-		ret = -EAGAIN;
-		goto out;
-	}
-
-	ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
-	if (ret < 0)
-		goto out;
-
-	if (ret > 0) {
-		ret = 0;
-		/* there are no items in the tree for us to truncate, we're
-		 * done
-		 */
-		if (path->slots[0] == 0)
-			goto out;
-		path->slots[0]--;
-	}
-
-	while (1) {
-		u64 clear_start = 0, clear_len = 0;
-
-		fi = NULL;
-		leaf = path->nodes[0];
-		btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
-		found_type = found_key.type;
-
-		if (found_key.objectid != ino)
-			break;
-
-		if (found_type < min_type)
-			break;
-
-		item_end = found_key.offset;
-		if (found_type == BTRFS_EXTENT_DATA_KEY) {
-			fi = btrfs_item_ptr(leaf, path->slots[0],
-					    struct btrfs_file_extent_item);
-			extent_type = btrfs_file_extent_type(leaf, fi);
-			if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
-				item_end +=
-				    btrfs_file_extent_num_bytes(leaf, fi);
-
-				trace_btrfs_truncate_show_fi_regular(
-					inode, leaf, fi, found_key.offset);
-			} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
-				item_end += btrfs_file_extent_ram_bytes(leaf,
-									fi);
-
-				trace_btrfs_truncate_show_fi_inline(
-					inode, leaf, fi, path->slots[0],
-					found_key.offset);
-			}
-			item_end--;
-		}
-		if (found_type > min_type) {
-			del_item = 1;
-		} else {
-			if (item_end < new_size)
-				break;
-			if (found_key.offset >= new_size)
-				del_item = 1;
-			else
-				del_item = 0;
-		}
-		found_extent = 0;
-		/* FIXME, shrink the extent if the ref count is only 1 */
-		if (found_type != BTRFS_EXTENT_DATA_KEY)
-			goto delete;
-
-		if (extents_found != NULL)
-			(*extents_found)++;
-
-		if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
-			u64 num_dec;
-
-			clear_start = found_key.offset;
-			extent_start = btrfs_file_extent_disk_bytenr(leaf, fi);
-			if (!del_item) {
-				u64 orig_num_bytes =
-					btrfs_file_extent_num_bytes(leaf, fi);
-				extent_num_bytes = ALIGN(new_size -
-						found_key.offset,
-						fs_info->sectorsize);
-				clear_start = ALIGN(new_size, fs_info->sectorsize);
-				btrfs_set_file_extent_num_bytes(leaf, fi,
-							 extent_num_bytes);
-				num_dec = (orig_num_bytes -
-					   extent_num_bytes);
-				if (test_bit(BTRFS_ROOT_SHAREABLE,
-					     &root->state) &&
-				    extent_start != 0)
-					inode_sub_bytes(&inode->vfs_inode,
-							num_dec);
-				btrfs_mark_buffer_dirty(leaf);
-			} else {
-				extent_num_bytes =
-					btrfs_file_extent_disk_num_bytes(leaf,
-									 fi);
-				extent_offset = found_key.offset -
-					btrfs_file_extent_offset(leaf, fi);
-
-				/* FIXME blocksize != 4096 */
-				num_dec = btrfs_file_extent_num_bytes(leaf, fi);
-				if (extent_start != 0) {
-					found_extent = 1;
-					if (test_bit(BTRFS_ROOT_SHAREABLE,
-						     &root->state))
-						inode_sub_bytes(&inode->vfs_inode,
-								num_dec);
-				}
-			}
-			clear_len = num_dec;
-		} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
-			/*
-			 * we can't truncate inline items that have had
-			 * special encodings
-			 */
-			if (!del_item &&
-			    btrfs_file_extent_encryption(leaf, fi) == 0 &&
-			    btrfs_file_extent_other_encoding(leaf, fi) == 0 &&
-			    btrfs_file_extent_compression(leaf, fi) == 0) {
-				u32 size = (u32)(new_size - found_key.offset);
-
-				btrfs_set_file_extent_ram_bytes(leaf, fi, size);
-				size = btrfs_file_extent_calc_inline_size(size);
-				btrfs_truncate_item(path, size, 1);
-			} else if (!del_item) {
-				/*
-				 * We have to bail so the last_size is set to
-				 * just before this extent.
-				 */
-				ret = NEED_TRUNCATE_BLOCK;
-				break;
-			} else {
-				/*
-				 * Inline extents are special, we just treat
-				 * them as a full sector worth in the file
-				 * extent tree just for simplicity sake.
-				 */
-				clear_len = fs_info->sectorsize;
-			}
-
-			if (test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
-				inode_sub_bytes(&inode->vfs_inode,
-						item_end + 1 - new_size);
-		}
-delete:
-		/*
-		 * We use btrfs_truncate_inode_items() to clean up log trees for
-		 * multiple fsyncs, and in this case we don't want to clear the
-		 * file extent range because it's just the log.
-		 */
-		if (root == inode->root) {
-			ret = btrfs_inode_clear_file_extent_range(inode,
-						  clear_start, clear_len);
-			if (ret) {
-				btrfs_abort_transaction(trans, ret);
-				break;
-			}
-		}
-
-		if (del_item)
-			last_size = found_key.offset;
-		else
-			last_size = new_size;
-		if (del_item) {
-			if (!pending_del_nr) {
-				/* no pending yet, add ourselves */
-				pending_del_slot = path->slots[0];
-				pending_del_nr = 1;
-			} else if (pending_del_nr &&
-				   path->slots[0] + 1 == pending_del_slot) {
-				/* hop on the pending chunk */
-				pending_del_nr++;
-				pending_del_slot = path->slots[0];
-			} else {
-				BUG();
-			}
-		} else {
-			break;
-		}
-		should_throttle = false;
-
-		if (found_extent &&
-		    root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
-			struct btrfs_ref ref = { 0 };
-
-			bytes_deleted += extent_num_bytes;
-
-			btrfs_init_generic_ref(&ref, BTRFS_DROP_DELAYED_REF,
-					extent_start, extent_num_bytes, 0);
-			btrfs_init_data_ref(&ref, btrfs_header_owner(leaf),
-					ino, extent_offset,
-					root->root_key.objectid, false);
-			ret = btrfs_free_extent(trans, &ref);
-			if (ret) {
-				btrfs_abort_transaction(trans, ret);
-				break;
-			}
-			if (be_nice) {
-				if (btrfs_should_throttle_delayed_refs(trans))
-					should_throttle = true;
-			}
-		}
-
-		if (found_type == BTRFS_INODE_ITEM_KEY)
-			break;
-
-		if (path->slots[0] == 0 ||
-		    path->slots[0] != pending_del_slot ||
-		    should_throttle) {
-			if (pending_del_nr) {
-				ret = btrfs_del_items(trans, root, path,
-						pending_del_slot,
-						pending_del_nr);
-				if (ret) {
-					btrfs_abort_transaction(trans, ret);
-					break;
-				}
-				pending_del_nr = 0;
-			}
-			btrfs_release_path(path);
-
-			/*
-			 * We can generate a lot of delayed refs, so we need to
-			 * throttle every once and a while and make sure we're
-			 * adding enough space to keep up with the work we are
-			 * generating.  Since we hold a transaction here we
-			 * can't flush, and we don't want to FLUSH_LIMIT because
-			 * we could have generated too many delayed refs to
-			 * actually allocate, so just bail if we're short and
-			 * let the normal reservation dance happen higher up.
-			 */
-			if (should_throttle) {
-				ret = btrfs_delayed_refs_rsv_refill(fs_info,
-							BTRFS_RESERVE_NO_FLUSH);
-				if (ret) {
-					ret = -EAGAIN;
-					break;
-				}
-			}
-			goto search_again;
-		} else {
-			path->slots[0]--;
-		}
-	}
-out:
-	if (ret >= 0 && pending_del_nr) {
-		int err;
-
-		err = btrfs_del_items(trans, root, path, pending_del_slot,
-				      pending_del_nr);
-		if (err) {
-			btrfs_abort_transaction(trans, err);
-			ret = err;
-		}
-	}
-	if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
-		ASSERT(last_size >= new_size);
-		if (!ret && last_size > new_size)
-			last_size = new_size;
-		btrfs_inode_safe_disk_i_size_write(inode, last_size);
-		unlock_extent_cached(&inode->io_tree, lock_start, (u64)-1,
-				     &cached_state);
-	}
-
-	btrfs_free_path(path);
-	return ret;
-}
-
 /*
  * btrfs_truncate_block - read, zero a chunk and write a block
  * @inode - inode that we're zeroing
@@ -8997,11 +8614,11 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 
 	/*
 	 * We can't call btrfs_truncate_block inside a trans handle as we could
-	 * deadlock with freeze, if we got NEED_TRUNCATE_BLOCK then we know
-	 * we've truncated everything except the last little bit, and can do
-	 * btrfs_truncate_block and then update the disk_i_size.
+	 * deadlock with freeze, if we got BTRFS_NEED_TRUNCATE_BLOCK then we
+	 * know we've truncated everything except the last little bit, and can
+	 * do btrfs_truncate_block and then update the disk_i_size.
 	 */
-	if (ret == NEED_TRUNCATE_BLOCK) {
+	if (ret == BTRFS_NEED_TRUNCATE_BLOCK) {
 		btrfs_end_transaction(trans);
 		btrfs_btree_balance_dirty(fs_info);
 
-- 
2.26.3

[PATCH 03/18] btrfs: move extent locking outside of btrfs_truncate_inode_items

[Josef Bacik]

Currently we are locking the extent and dropping the extent cache for
any inodes we truncate, unless they're in the tree log.  We call this
helper from

- truncate
- evict
- tree log
- free space cache truncation

For evict we've already dropped all of the extent cache for this inode
once we've gotten here, and we're the only one accessing this inode, so
this step is unnecessary.

For the tree log code we already skip this part.

Pull this work into the truncate path and the free space cache
truncation path.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c | 20 +++++++++++++-------
 fs/btrfs/inode-item.c       | 18 ------------------
 fs/btrfs/inode.c            | 18 ++++++++++++++++++
 3 files changed, 31 insertions(+), 25 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 55e1be703a39..28b9c63ba536 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -289,9 +289,11 @@ int btrfs_check_trunc_cache_free_space(struct btrfs_fs_info *fs_info,
 
 int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 				    struct btrfs_block_group *block_group,
-				    struct inode *inode)
+				    struct inode *vfs_inode)
 {
-	struct btrfs_root *root = BTRFS_I(inode)->root;
+	struct btrfs_inode *inode = BTRFS_I(vfs_inode);
+	struct btrfs_root *root = inode->root;
+	struct extent_state *cached_state = NULL;
 	int ret = 0;
 	bool locked = false;
 
@@ -321,19 +323,23 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 		btrfs_free_path(path);
 	}
 
-	btrfs_i_size_write(BTRFS_I(inode), 0);
-	truncate_pagecache(inode, 0);
+	btrfs_i_size_write(inode, 0);
+	truncate_pagecache(vfs_inode, 0);
+
+	lock_extent_bits(&inode->io_tree, 0, (u64)-1, &cached_state);
+	btrfs_drop_extent_cache(inode, 0, (u64)-1, 0);
 
 	/*
 	 * We skip the throttling logic for free space cache inodes, so we don't
 	 * need to check for -EAGAIN.
 	 */
-	ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
-					 0, BTRFS_EXTENT_DATA_KEY, NULL);
+	ret = btrfs_truncate_inode_items(trans, root, inode, 0,
+					 BTRFS_EXTENT_DATA_KEY, NULL);
+	unlock_extent_cached(&inode->io_tree, 0, (u64)-1, &cached_state);
 	if (ret)
 		goto fail;
 
-	ret = btrfs_update_inode(trans, root, BTRFS_I(inode));
+	ret = btrfs_update_inode(trans, root, inode);
 
 fail:
 	if (locked)
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 6e4b244ce96c..124f952ab9f2 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -473,8 +473,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	u64 bytes_deleted = 0;
 	bool be_nice = false;
 	bool should_throttle = false;
-	const u64 lock_start = ALIGN_DOWN(new_size, fs_info->sectorsize);
-	struct extent_state *cached_state = NULL;
 
 	BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY);
 
@@ -492,20 +490,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		return -ENOMEM;
 	path->reada = READA_BACK;
 
-	if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
-		lock_extent_bits(&inode->io_tree, lock_start, (u64)-1,
-				 &cached_state);
-
-		/*
-		 * We want to drop from the next block forward in case this
-		 * new size is not block aligned since we will be keeping the
-		 * last block of the extent just the way it is.
-		 */
-		btrfs_drop_extent_cache(inode, ALIGN(new_size,
-					fs_info->sectorsize),
-					(u64)-1, 0);
-	}
-
 	/*
 	 * This function is also used to drop the items in the log tree before
 	 * we relog the inode, so if root != BTRFS_I(inode)->root, it means
@@ -788,8 +772,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		if (!ret && last_size > new_size)
 			last_size = new_size;
 		btrfs_inode_safe_disk_i_size_write(inode, last_size);
-		unlock_extent_cached(&inode->io_tree, lock_start, (u64)-1,
-				     &cached_state);
 	}
 
 	btrfs_free_path(path);
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index c29e7c87ff27..da474791da23 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -8583,10 +8583,28 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 	trans->block_rsv = rsv;
 
 	while (1) {
+		struct extent_state *cached_state = NULL;
+		const u64 new_size = inode->i_size;
+		const u64 lock_start = ALIGN_DOWN(new_size, fs_info->sectorsize);
+
+		lock_extent_bits(&BTRFS_I(inode)->io_tree, lock_start, (u64)-1,
+				 &cached_state);
+		/*
+		 * We want to drop from the next block forward in case this
+		 * new size is not block aligned since we will be keeping the
+		 * last block of the extent just the way it is.
+		 */
+		btrfs_drop_extent_cache(BTRFS_I(inode),
+					ALIGN(new_size, fs_info->sectorsize),
+					(u64)-1, 0);
+
 		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
 						 inode->i_size,
 						 BTRFS_EXTENT_DATA_KEY,
 						 &extents_found);
+		unlock_extent_cached(&BTRFS_I(inode)->io_tree, lock_start,
+				     (u64)-1, &cached_state);
+
 		trans->block_rsv = &fs_info->trans_block_rsv;
 		if (ret != -ENOSPC && ret != -EAGAIN)
 			break;
-- 
2.26.3

[PATCH 04/18] btrfs: remove free space cache inode check in btrfs_truncate_inode_items

[Josef Bacik]

We no longer have inode cache, so this check is extraneous as the only
inode cache is in the tree_root, which is not marked as SHAREABLE.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 124f952ab9f2..ee7ac75ed6c8 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -477,12 +477,10 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY);
 
 	/*
-	 * For non-free space inodes and non-shareable roots, we want to back
-	 * off from time to time.  This means all inodes in subvolume roots,
-	 * reloc roots, and data reloc roots.
+	 * For shareable roots we want to back off from time to time, this turns
+	 * out to be subvolume roots, reloc roots, and data reloc roots.
 	 */
-	if (!btrfs_is_free_space_inode(inode) &&
-	    test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
+	if (test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
 		be_nice = true;
 
 	path = btrfs_alloc_path();
-- 
2.26.3

[PATCH 05/18] btrfs: move btrfs_kill_delayed_inode_items into evict

[Josef Bacik]

We have a special case in btrfs_truncate_inode_items() to call
btrfs_kill_delayed_inode_items() if min_type == 0, which is only called
during evict.

Instead move this out into evict proper, and add some comments because I
erroneously attempted to remove this code altogether without
understanding what we were doing.

Evict is updating the inode only because we only care about making sure
the i_nlink count has hit disk.  If we had pending deletions we don't
want to process those via the delayed inode updates, we simply want to
drop all of them and reclaim the reserved metadata space.  Then from
there the btrfs_truncate_inode_items() will do the work to remove all of
the items as appropriate.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c |  9 ---------
 fs/btrfs/inode.c      | 12 ++++++++++++
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index ee7ac75ed6c8..bc59f80510ad 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -488,15 +488,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		return -ENOMEM;
 	path->reada = READA_BACK;
 
-	/*
-	 * This function is also used to drop the items in the log tree before
-	 * we relog the inode, so if root != BTRFS_I(inode)->root, it means
-	 * it is used to drop the logged items. So we shouldn't kill the delayed
-	 * items.
-	 */
-	if (min_type == 0 && root == inode->root)
-		btrfs_kill_delayed_inode_items(inode);
-
 	key.objectid = ino;
 	key.offset = (u64)-1;
 	key.type = (u8)-1;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index da474791da23..73bb7acf6813 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -5226,10 +5226,22 @@ void btrfs_evict_inode(struct inode *inode)
 		goto no_delete;
 	}
 
+	/*
+	 * This makes sure the inode item in tree is uptodate and the space for
+	 * the inode update is released.
+	 */
 	ret = btrfs_commit_inode_delayed_inode(BTRFS_I(inode));
 	if (ret)
 		goto no_delete;
 
+	/*
+	 * This drops any pending insert or delete operations we have for this
+	 * inode.  We could have a delayed dir index deletion queued up, but
+	 * we're removing the inode completely so that'll be taken care of in
+	 * the truncate.
+	 */
+	btrfs_kill_delayed_inode_items(BTRFS_I(inode));
+
 	rsv = btrfs_alloc_block_rsv(fs_info, BTRFS_BLOCK_RSV_TEMP);
 	if (!rsv)
 		goto no_delete;
-- 
2.26.3

[PATCH 06/18] btrfs: remove found_extent from btrfs_truncate_inode_items

[Josef Bacik]

We only set this if we find a normal file extent, del_item == 1, and the
file extent points to a real extent and isn't a hole extent.  We can use
del_item == 1 && extent_start != 0 to get the same information that
found_extent provides, so remove this variable and use the other
variables instead.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index bc59f80510ad..8afc8d1c607b 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -457,13 +457,11 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	struct btrfs_file_extent_item *fi;
 	struct btrfs_key key;
 	struct btrfs_key found_key;
-	u64 extent_start = 0;
 	u64 extent_num_bytes = 0;
 	u64 extent_offset = 0;
 	u64 item_end = 0;
 	u64 last_size = new_size;
 	u32 found_type = (u8)-1;
-	int found_extent;
 	int del_item;
 	int pending_del_nr = 0;
 	int pending_del_slot = 0;
@@ -519,7 +517,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	}
 
 	while (1) {
-		u64 clear_start = 0, clear_len = 0;
+		u64 clear_start = 0, clear_len = 0, extent_start = 0;
 
 		fi = NULL;
 		leaf = path->nodes[0];
@@ -563,7 +561,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			else
 				del_item = 0;
 		}
-		found_extent = 0;
+
 		/* FIXME, shrink the extent if the ref count is only 1 */
 		if (found_type != BTRFS_EXTENT_DATA_KEY)
 			goto delete;
@@ -603,7 +601,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 				/* FIXME blocksize != 4096 */
 				num_dec = btrfs_file_extent_num_bytes(leaf, fi);
 				if (extent_start != 0) {
-					found_extent = 1;
 					if (test_bit(BTRFS_ROOT_SHAREABLE,
 						     &root->state))
 						inode_sub_bytes(&inode->vfs_inode,
@@ -682,7 +679,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		}
 		should_throttle = false;
 
-		if (found_extent &&
+		if (del_item && extent_start != 0 &&
 		    root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
 			struct btrfs_ref ref = { 0 };
 
-- 
2.26.3

[PATCH 07/18] btrfs: add btrfs_truncate_control struct

[Josef Bacik]

I'm going to be adding more arguments and counters to
btrfs_truncate_inode_items, so add a control struct to handle all of the
extra arguments to make it easier to follow.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c |  7 +++++--
 fs/btrfs/inode-item.c       | 25 ++++++++-----------------
 fs/btrfs/inode-item.h       | 18 ++++++++++++++++--
 fs/btrfs/inode.c            | 18 ++++++++++++------
 fs/btrfs/tree-log.c         |  6 +++++-
 5 files changed, 46 insertions(+), 28 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 28b9c63ba536..a05dd3d29695 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -291,6 +291,10 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 				    struct btrfs_block_group *block_group,
 				    struct inode *vfs_inode)
 {
+	struct btrfs_truncate_control control = {
+		.new_size = 0,
+		.min_type = BTRFS_EXTENT_DATA_KEY,
+	};
 	struct btrfs_inode *inode = BTRFS_I(vfs_inode);
 	struct btrfs_root *root = inode->root;
 	struct extent_state *cached_state = NULL;
@@ -333,8 +337,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 	 * We skip the throttling logic for free space cache inodes, so we don't
 	 * need to check for -EAGAIN.
 	 */
-	ret = btrfs_truncate_inode_items(trans, root, inode, 0,
-					 BTRFS_EXTENT_DATA_KEY, NULL);
+	ret = btrfs_truncate_inode_items(trans, root, inode, &control);
 	unlock_extent_cached(&inode->io_tree, 0, (u64)-1, &cached_state);
 	if (ret)
 		goto fail;
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 8afc8d1c607b..fa172c760fe2 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -425,16 +425,8 @@ int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
  * @trans:		A transaction handle.
  * @root:		The root from which to remove items.
  * @inode:		The inode whose items we want to remove.
- * @new_size:		The new i_size for the inode. This is only applicable when
- *			@min_type is BTRFS_EXTENT_DATA_KEY, must be 0 otherwise.
- * @min_type:		The minimum key type to remove. All keys with a type
- *			greater than this value are removed and all keys with
- *			this type are removed only if their offset is >= @new_size.
- * @extents_found:	Output parameter that will contain the number of file
- *			extent items that were removed or adjusted to the new
- *			inode i_size. The caller is responsible for initializing
- *			the counter. Also, it can be NULL if the caller does not
- *			need this counter.
+ * @control:		The btrfs_truncate_control to control how and what we
+ *			are truncating.
  *
  * Remove all keys associated with the inode from the given root that have a key
  * with a type greater than or equals to @min_type. When @min_type has a value of
@@ -448,8 +440,7 @@ int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
 int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root,
 			       struct btrfs_inode *inode,
-			       u64 new_size, u32 min_type,
-			       u64 *extents_found)
+			       struct btrfs_truncate_control *control)
 {
 	struct btrfs_fs_info *fs_info = root->fs_info;
 	struct btrfs_path *path;
@@ -457,6 +448,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	struct btrfs_file_extent_item *fi;
 	struct btrfs_key key;
 	struct btrfs_key found_key;
+	u64 new_size = control->new_size;
 	u64 extent_num_bytes = 0;
 	u64 extent_offset = 0;
 	u64 item_end = 0;
@@ -472,7 +464,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	bool be_nice = false;
 	bool should_throttle = false;
 
-	BUG_ON(new_size > 0 && min_type != BTRFS_EXTENT_DATA_KEY);
+	BUG_ON(new_size > 0 && control->min_type != BTRFS_EXTENT_DATA_KEY);
 
 	/*
 	 * For shareable roots we want to back off from time to time, this turns
@@ -527,7 +519,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		if (found_key.objectid != ino)
 			break;
 
-		if (found_type < min_type)
+		if (found_type < control->min_type)
 			break;
 
 		item_end = found_key.offset;
@@ -551,7 +543,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			}
 			item_end--;
 		}
-		if (found_type > min_type) {
+		if (found_type > control->min_type) {
 			del_item = 1;
 		} else {
 			if (item_end < new_size)
@@ -566,8 +558,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		if (found_type != BTRFS_EXTENT_DATA_KEY)
 			goto delete;
 
-		if (extents_found != NULL)
-			(*extents_found)++;
+		control->extents_found++;
 
 		if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
 			u64 num_dec;
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 63e8f45f110f..47c3fec579f8 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -9,10 +9,24 @@
  */
 #define BTRFS_NEED_TRUNCATE_BLOCK 1
 
+struct btrfs_truncate_control {
+	/* IN: the size we're truncating to. */
+	u64 new_size;
+
+	/* OUT: the number of extents truncated. */
+	u64 extents_found;
+
+	/*
+	 * IN: minimum key type to remove.  All key types with this type are
+	 * removed only if their offset >= new_size.
+	 */
+	u32 min_type;
+};
+
 int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root,
-			       struct btrfs_inode *inode, u64 new_size,
-			       u32 min_type, u64 *extents_found);
+			       struct btrfs_inode *inode,
+			       struct btrfs_truncate_control *control);
 int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
 			   struct btrfs_root *root,
 			   const char *name, int name_len,
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 73bb7acf6813..6600c474b2e8 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -5251,6 +5251,11 @@ void btrfs_evict_inode(struct inode *inode)
 	btrfs_i_size_write(BTRFS_I(inode), 0);
 
 	while (1) {
+		struct btrfs_truncate_control control = {
+			.new_size = 0,
+			.min_type = 0,
+		};
+
 		trans = evict_refill_and_join(root, rsv);
 		if (IS_ERR(trans))
 			goto free_rsv;
@@ -5258,7 +5263,7 @@ void btrfs_evict_inode(struct inode *inode)
 		trans->block_rsv = rsv;
 
 		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
-						 0, 0, NULL);
+						 &control);
 		trans->block_rsv = &fs_info->trans_block_rsv;
 		btrfs_end_transaction(trans);
 		btrfs_btree_balance_dirty(fs_info);
@@ -8527,6 +8532,9 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)
 
 static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 {
+	struct btrfs_truncate_control control = {
+		.min_type = BTRFS_EXTENT_DATA_KEY,
+	};
 	struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
 	struct btrfs_root *root = BTRFS_I(inode)->root;
 	struct btrfs_block_rsv *rsv;
@@ -8534,7 +8542,6 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 	struct btrfs_trans_handle *trans;
 	u64 mask = fs_info->sectorsize - 1;
 	u64 min_size = btrfs_calc_metadata_size(fs_info, 1);
-	u64 extents_found = 0;
 
 	if (!skip_writeback) {
 		ret = btrfs_wait_ordered_range(inode, inode->i_size & (~mask),
@@ -8599,6 +8606,7 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 		const u64 new_size = inode->i_size;
 		const u64 lock_start = ALIGN_DOWN(new_size, fs_info->sectorsize);
 
+		control.new_size = new_size;
 		lock_extent_bits(&BTRFS_I(inode)->io_tree, lock_start, (u64)-1,
 				 &cached_state);
 		/*
@@ -8611,9 +8619,7 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 					(u64)-1, 0);
 
 		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
-						 inode->i_size,
-						 BTRFS_EXTENT_DATA_KEY,
-						 &extents_found);
+						 &control);
 		unlock_extent_cached(&BTRFS_I(inode)->io_tree, lock_start,
 				     (u64)-1, &cached_state);
 
@@ -8692,7 +8698,7 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 	 * between the old i_size and the new i_size, and there were no prealloc
 	 * extents beyond i_size to drop.
 	 */
-	if (extents_found > 0)
+	if (control.extents_found > 0)
 		set_bit(BTRFS_INODE_NEEDS_FULL_SYNC, &BTRFS_I(inode)->runtime_flags);
 
 	return ret;
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index c7a7f78708d5..04374a7346db 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -4095,11 +4095,15 @@ static int truncate_inode_items(struct btrfs_trans_handle *trans,
 				struct btrfs_inode *inode,
 				u64 new_size, u32 min_type)
 {
+	struct btrfs_truncate_control control = {
+		.new_size = new_size,
+		.min_type = min_type,
+	};
 	int ret;
 
 	do {
 		ret = btrfs_truncate_inode_items(trans, log_root, inode,
-						 new_size, min_type, NULL);
+						 &control);
 	} while (ret == -EAGAIN);
 
 	return ret;
-- 
2.26.3

[PATCH 08/18] btrfs: only update i_size in truncate paths that care

[Josef Bacik]

We currently will update the i_size of the inode as we truncate it down,
however we skip this if we're calling btrfs_truncate_inode_items from
the tree log code.  However we also don't care about this in the case of
evict.  Instead keep track of this value in the btrfs_truncate_control
and then have btrfs_truncate() and the free space cache truncate path
both do the i_size update themselves.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c |  3 +++
 fs/btrfs/inode-item.c       | 17 ++++++++---------
 fs/btrfs/inode-item.h       |  3 +++
 fs/btrfs/inode.c            |  4 ++++
 4 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index a05dd3d29695..fd469beb0985 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -338,6 +338,9 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 	 * need to check for -EAGAIN.
 	 */
 	ret = btrfs_truncate_inode_items(trans, root, inode, &control);
+
+	btrfs_inode_safe_disk_i_size_write(inode, control.last_size);
+
 	unlock_extent_cached(&inode->io_tree, 0, (u64)-1, &cached_state);
 	if (ret)
 		goto fail;
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index fa172c760fe2..15dc5352d08a 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -452,7 +452,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	u64 extent_num_bytes = 0;
 	u64 extent_offset = 0;
 	u64 item_end = 0;
-	u64 last_size = new_size;
 	u32 found_type = (u8)-1;
 	int del_item;
 	int pending_del_nr = 0;
@@ -466,6 +465,8 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 
 	BUG_ON(new_size > 0 && control->min_type != BTRFS_EXTENT_DATA_KEY);
 
+	control->last_size = new_size;
+
 	/*
 	 * For shareable roots we want to back off from time to time, this turns
 	 * out to be subvolume roots, reloc roots, and data reloc roots.
@@ -649,9 +650,9 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		}
 
 		if (del_item)
-			last_size = found_key.offset;
+			control->last_size = found_key.offset;
 		else
-			last_size = new_size;
+			control->last_size = new_size;
 		if (del_item) {
 			if (!pending_del_nr) {
 				/* no pending yet, add ourselves */
@@ -744,12 +745,10 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			ret = err;
 		}
 	}
-	if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
-		ASSERT(last_size >= new_size);
-		if (!ret && last_size > new_size)
-			last_size = new_size;
-		btrfs_inode_safe_disk_i_size_write(inode, last_size);
-	}
+
+	ASSERT(control->last_size  >= new_size);
+	if (!ret && control->last_size > new_size)
+		control->last_size = new_size;
 
 	btrfs_free_path(path);
 	return ret;
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 47c3fec579f8..21adab1df4e5 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -16,6 +16,9 @@ struct btrfs_truncate_control {
 	/* OUT: the number of extents truncated. */
 	u64 extents_found;
 
+	/* OUT: the last size we truncated this inode to. */
+	u64 last_size;
+
 	/*
 	 * IN: minimum key type to remove.  All key types with this type are
 	 * removed only if their offset >= new_size.
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 6600c474b2e8..23b47c7bce0f 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -8620,6 +8620,10 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 
 		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
 						 &control);
+
+		btrfs_inode_safe_disk_i_size_write(BTRFS_I(inode),
+						   control.last_size);
+
 		unlock_extent_cached(&BTRFS_I(inode)->io_tree, lock_start,
 				     (u64)-1, &cached_state);
 
-- 
2.26.3

[PATCH 09/18] btrfs: only call inode_sub_bytes in truncate paths that care

[Josef Bacik]

We currently have a bunch of awkward checks to make sure we only update
the inode i_bytes if we're truncating the real inode.  Instead keep
track of the number of bytes we need to sub in the
btrfs_truncate_control, and then do the appropriate adjustment in the
truncate paths that care.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c |  1 +
 fs/btrfs/inode-item.c       | 20 ++++++--------------
 fs/btrfs/inode-item.h       |  3 +++
 fs/btrfs/inode.c            |  1 +
 4 files changed, 11 insertions(+), 14 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index fd469beb0985..d2f4716f8485 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -339,6 +339,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 	 */
 	ret = btrfs_truncate_inode_items(trans, root, inode, &control);
 
+	inode_sub_bytes(&inode->vfs_inode, control.sub_bytes);
 	btrfs_inode_safe_disk_i_size_write(inode, control.last_size);
 
 	unlock_extent_cached(&inode->io_tree, 0, (u64)-1, &cached_state);
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 15dc5352d08a..ebbe4054ae93 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -466,6 +466,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	BUG_ON(new_size > 0 && control->min_type != BTRFS_EXTENT_DATA_KEY);
 
 	control->last_size = new_size;
+	control->sub_bytes = 0;
 
 	/*
 	 * For shareable roots we want to back off from time to time, this turns
@@ -577,11 +578,8 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 							 extent_num_bytes);
 				num_dec = (orig_num_bytes -
 					   extent_num_bytes);
-				if (test_bit(BTRFS_ROOT_SHAREABLE,
-					     &root->state) &&
-				    extent_start != 0)
-					inode_sub_bytes(&inode->vfs_inode,
-							num_dec);
+				if (extent_start != 0)
+					control->sub_bytes += num_dec;
 				btrfs_mark_buffer_dirty(leaf);
 			} else {
 				extent_num_bytes =
@@ -592,12 +590,8 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 
 				/* FIXME blocksize != 4096 */
 				num_dec = btrfs_file_extent_num_bytes(leaf, fi);
-				if (extent_start != 0) {
-					if (test_bit(BTRFS_ROOT_SHAREABLE,
-						     &root->state))
-						inode_sub_bytes(&inode->vfs_inode,
-								num_dec);
-				}
+				if (extent_start != 0)
+					control->sub_bytes += num_dec;
 			}
 			clear_len = num_dec;
 		} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
@@ -630,9 +624,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 				clear_len = fs_info->sectorsize;
 			}
 
-			if (test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
-				inode_sub_bytes(&inode->vfs_inode,
-						item_end + 1 - new_size);
+			control->sub_bytes += item_end + 1 - new_size;
 		}
 delete:
 		/*
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 21adab1df4e5..771e264a2ede 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -19,6 +19,9 @@ struct btrfs_truncate_control {
 	/* OUT: the last size we truncated this inode to. */
 	u64 last_size;
 
+	/* OUT: the number of bytes to sub from this inode. */
+	u64 sub_bytes;
+
 	/*
 	 * IN: minimum key type to remove.  All key types with this type are
 	 * removed only if their offset >= new_size.
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 23b47c7bce0f..306d410b4db4 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -8621,6 +8621,7 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
 						 &control);
 
+		inode_sub_bytes(inode, control.sub_bytes);
 		btrfs_inode_safe_disk_i_size_write(BTRFS_I(inode),
 						   control.last_size);
 
-- 
2.26.3

[PATCH 10/18] btrfs: control extent reference updates with a control flag for truncate

[Josef Bacik]

We've had weird bugs in the past where we forgot to adjust the truncate
path to deal with the fact that we can be called by the tree log path.
Instead of checking if our root is a LOG_ROOT use a flag on the
btrfs_truncate_control to indicate that we don't want to do extent
reference updates during this truncate.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 3 +--
 fs/btrfs/inode-item.h | 6 ++++++
 fs/btrfs/tree-log.c   | 1 +
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index ebbe4054ae93..79305d646b98 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -663,8 +663,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		}
 		should_throttle = false;
 
-		if (del_item && extent_start != 0 &&
-		    root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
+		if (del_item && extent_start != 0 && !control->skip_ref_updates) {
 			struct btrfs_ref ref = { 0 };
 
 			bytes_deleted += extent_num_bytes;
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 771e264a2ede..0cb16cac07d1 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -27,6 +27,12 @@ struct btrfs_truncate_control {
 	 * removed only if their offset >= new_size.
 	 */
 	u32 min_type;
+
+	/*
+	 * IN: true if we don't want to do extent reference updates for any file
+	 * extents we drop.
+	 */
+	bool skip_ref_updates;
 };
 
 int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index 04374a7346db..11b9b516af80 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -4098,6 +4098,7 @@ static int truncate_inode_items(struct btrfs_trans_handle *trans,
 	struct btrfs_truncate_control control = {
 		.new_size = new_size,
 		.min_type = min_type,
+		.skip_ref_updates = true,
 	};
 	int ret;
 
-- 
2.26.3

[PATCH 11/18] btrfs: use a flag to control when to clear the file extent range

[Josef Bacik]

We only care about updating the file extent range when we are doing a
normal truncation.  We skip this for tree logging currently, but we can
also skip this for eviction as well.  Using a flag makes it more
explicit when we want to do this work.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c | 1 +
 fs/btrfs/inode-item.c       | 8 ++++----
 fs/btrfs/inode-item.h       | 6 ++++++
 fs/btrfs/inode.c            | 1 +
 4 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index d2f4716f8485..3a6bf361409b 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -294,6 +294,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 	struct btrfs_truncate_control control = {
 		.new_size = 0,
 		.min_type = BTRFS_EXTENT_DATA_KEY,
+		.clear_extent_range = true,
 	};
 	struct btrfs_inode *inode = BTRFS_I(vfs_inode);
 	struct btrfs_root *root = inode->root;
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 79305d646b98..225a5cd3f0ea 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -628,11 +628,11 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		}
 delete:
 		/*
-		 * We use btrfs_truncate_inode_items() to clean up log trees for
-		 * multiple fsyncs, and in this case we don't want to clear the
-		 * file extent range because it's just the log.
+		 * We only want to clear the file extent range if we're
+		 * modifying the actual inode's mapping, which is just the
+		 * normal truncate path.
 		 */
-		if (root == inode->root) {
+		if (control->clear_extent_range) {
 			ret = btrfs_inode_clear_file_extent_range(inode,
 						  clear_start, clear_len);
 			if (ret) {
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 0cb16cac07d1..50acee3f4e28 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -33,6 +33,12 @@ struct btrfs_truncate_control {
 	 * extents we drop.
 	 */
 	bool skip_ref_updates;
+
+	/*
+	 * IN: true if we need to clear the file extent range for the inode as
+	 * we drop the file extent items.
+	 */
+	bool clear_extent_range;
 };
 
 int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 306d410b4db4..45dc4355102a 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -8534,6 +8534,7 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 {
 	struct btrfs_truncate_control control = {
 		.min_type = BTRFS_EXTENT_DATA_KEY,
+		.clear_extent_range = true,
 	};
 	struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
 	struct btrfs_root *root = BTRFS_I(inode)->root;
-- 
2.26.3

[PATCH 12/18] btrfs: pass the ino via btrfs_truncate_control

[Josef Bacik]

In the future we are going to want to truncate inode items without
needing to have an btrfs_inode to pass in, so add ino to the
btrfs_truncate_control and use that to look up the inode items to
truncate.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c | 1 +
 fs/btrfs/inode-item.c       | 7 +++----
 fs/btrfs/inode-item.h       | 3 +++
 fs/btrfs/inode.c            | 2 ++
 fs/btrfs/tree-log.c         | 1 +
 5 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 3a6bf361409b..c2a34179bddc 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -293,6 +293,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 {
 	struct btrfs_truncate_control control = {
 		.new_size = 0,
+		.ino = btrfs_ino(BTRFS_I(vfs_inode)),
 		.min_type = BTRFS_EXTENT_DATA_KEY,
 		.clear_extent_range = true,
 	};
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 225a5cd3f0ea..b11c3da680fd 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -458,7 +458,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	int pending_del_slot = 0;
 	int extent_type = -1;
 	int ret;
-	u64 ino = btrfs_ino(inode);
 	u64 bytes_deleted = 0;
 	bool be_nice = false;
 	bool should_throttle = false;
@@ -480,7 +479,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		return -ENOMEM;
 	path->reada = READA_BACK;
 
-	key.objectid = ino;
+	key.objectid = control->ino;
 	key.offset = (u64)-1;
 	key.type = (u8)-1;
 
@@ -518,7 +517,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
 		found_type = found_key.type;
 
-		if (found_key.objectid != ino)
+		if (found_key.objectid != control->ino)
 			break;
 
 		if (found_type < control->min_type)
@@ -671,7 +670,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			btrfs_init_generic_ref(&ref, BTRFS_DROP_DELAYED_REF,
 					extent_start, extent_num_bytes, 0);
 			btrfs_init_data_ref(&ref, btrfs_header_owner(leaf),
-					ino, extent_offset,
+					control->ino, extent_offset,
 					root->root_key.objectid, false);
 			ret = btrfs_free_extent(trans, &ref);
 			if (ret) {
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 50acee3f4e28..1948461e5a46 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -22,6 +22,9 @@ struct btrfs_truncate_control {
 	/* OUT: the number of bytes to sub from this inode. */
 	u64 sub_bytes;
 
+	/* IN: the ino we are truncating. */
+	u64 ino;
+
 	/*
 	 * IN: minimum key type to remove.  All key types with this type are
 	 * removed only if their offset >= new_size.
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 45dc4355102a..06e7c5e26a65 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -5252,6 +5252,7 @@ void btrfs_evict_inode(struct inode *inode)
 
 	while (1) {
 		struct btrfs_truncate_control control = {
+			.ino = btrfs_ino(BTRFS_I(inode)),
 			.new_size = 0,
 			.min_type = 0,
 		};
@@ -8533,6 +8534,7 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)
 static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 {
 	struct btrfs_truncate_control control = {
+		.ino = btrfs_ino(BTRFS_I(inode)),
 		.min_type = BTRFS_EXTENT_DATA_KEY,
 		.clear_extent_range = true,
 	};
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index 11b9b516af80..fb3bf7cc21c5 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -4097,6 +4097,7 @@ static int truncate_inode_items(struct btrfs_trans_handle *trans,
 {
 	struct btrfs_truncate_control control = {
 		.new_size = new_size,
+		.ino = btrfs_ino(inode),
 		.min_type = min_type,
 		.skip_ref_updates = true,
 	};
-- 
2.26.3

[PATCH 13/18] btrfs: add inode to btrfs_truncate_control

[Josef Bacik]

In the future we're going to want to use btrfs_truncate_inode_items
without looking up the associated inode.  In order to accommodate this
add the inode to btrfs_truncate_control and handle the case where
control->inode is NULL appropriately.  This is fairly straightforward,
we simply need to add a helper for the trace points, as the file extent
map update is controlled by a flag on btrfs_truncate_control.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/free-space-cache.c |  3 ++-
 fs/btrfs/inode-item.c       | 32 +++++++++++++++++++++-----------
 fs/btrfs/inode-item.h       |  7 ++++++-
 fs/btrfs/inode.c            |  8 ++++----
 fs/btrfs/tree-log.c         |  3 +--
 5 files changed, 34 insertions(+), 19 deletions(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index c2a34179bddc..01a408db5683 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -292,6 +292,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 				    struct inode *vfs_inode)
 {
 	struct btrfs_truncate_control control = {
+		.inode = BTRFS_I(vfs_inode),
 		.new_size = 0,
 		.ino = btrfs_ino(BTRFS_I(vfs_inode)),
 		.min_type = BTRFS_EXTENT_DATA_KEY,
@@ -339,7 +340,7 @@ int btrfs_truncate_free_space_cache(struct btrfs_trans_handle *trans,
 	 * We skip the throttling logic for free space cache inodes, so we don't
 	 * need to check for -EAGAIN.
 	 */
-	ret = btrfs_truncate_inode_items(trans, root, inode, &control);
+	ret = btrfs_truncate_inode_items(trans, root, &control);
 
 	inode_sub_bytes(&inode->vfs_inode, control.sub_bytes);
 	btrfs_inode_safe_disk_i_size_write(inode, control.last_size);
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index b11c3da680fd..aee374e18131 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -419,6 +419,20 @@ int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
 	return ret;
 }
 
+static inline void btrfs_trace_truncate(struct btrfs_inode *inode,
+					struct extent_buffer *leaf,
+					struct btrfs_file_extent_item *fi,
+					u64 offset, int extent_type, int slot)
+{
+	if (!inode)
+		return;
+	if (extent_type == BTRFS_FILE_EXTENT_INLINE)
+		trace_btrfs_truncate_show_fi_inline(inode, leaf, fi, slot,
+						    offset);
+	else
+		trace_btrfs_truncate_show_fi_regular(inode, leaf, fi, offset);
+}
+
 /*
  * Remove inode items from a given root.
  *
@@ -439,7 +453,6 @@ int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
  */
 int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root,
-			       struct btrfs_inode *inode,
 			       struct btrfs_truncate_control *control)
 {
 	struct btrfs_fs_info *fs_info = root->fs_info;
@@ -462,6 +475,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	bool be_nice = false;
 	bool should_throttle = false;
 
+	ASSERT(control->inode || !control->clear_extent_range);
 	BUG_ON(new_size > 0 && control->min_type != BTRFS_EXTENT_DATA_KEY);
 
 	control->last_size = new_size;
@@ -528,20 +542,16 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			fi = btrfs_item_ptr(leaf, path->slots[0],
 					    struct btrfs_file_extent_item);
 			extent_type = btrfs_file_extent_type(leaf, fi);
-			if (extent_type != BTRFS_FILE_EXTENT_INLINE) {
+			if (extent_type != BTRFS_FILE_EXTENT_INLINE)
 				item_end +=
 				    btrfs_file_extent_num_bytes(leaf, fi);
-
-				trace_btrfs_truncate_show_fi_regular(
-					inode, leaf, fi, found_key.offset);
-			} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
+			else if (extent_type == BTRFS_FILE_EXTENT_INLINE)
 				item_end += btrfs_file_extent_ram_bytes(leaf,
 									fi);
 
-				trace_btrfs_truncate_show_fi_inline(
-					inode, leaf, fi, path->slots[0],
-					found_key.offset);
-			}
+			btrfs_trace_truncate(control->inode, leaf, fi,
+					     found_key.offset, extent_type,
+					     path->slots[0]);
 			item_end--;
 		}
 		if (found_type > control->min_type) {
@@ -632,7 +642,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		 * normal truncate path.
 		 */
 		if (control->clear_extent_range) {
-			ret = btrfs_inode_clear_file_extent_range(inode,
+			ret = btrfs_inode_clear_file_extent_range(control->inode,
 						  clear_start, clear_len);
 			if (ret) {
 				btrfs_abort_transaction(trans, ret);
diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
index 1948461e5a46..b8c788f71c95 100644
--- a/fs/btrfs/inode-item.h
+++ b/fs/btrfs/inode-item.h
@@ -10,6 +10,12 @@
 #define BTRFS_NEED_TRUNCATE_BLOCK 1
 
 struct btrfs_truncate_control {
+	/*
+	 * IN: the inode we're operating on, this can be NULL if
+	 * ->clear_extent_range is false.
+	 */
+	struct btrfs_inode *inode;
+
 	/* IN: the size we're truncating to. */
 	u64 new_size;
 
@@ -46,7 +52,6 @@ struct btrfs_truncate_control {
 
 int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root,
-			       struct btrfs_inode *inode,
 			       struct btrfs_truncate_control *control);
 int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
 			   struct btrfs_root *root,
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 06e7c5e26a65..07e8539819b9 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -5252,6 +5252,7 @@ void btrfs_evict_inode(struct inode *inode)
 
 	while (1) {
 		struct btrfs_truncate_control control = {
+			.inode = BTRFS_I(inode),
 			.ino = btrfs_ino(BTRFS_I(inode)),
 			.new_size = 0,
 			.min_type = 0,
@@ -5263,8 +5264,7 @@ void btrfs_evict_inode(struct inode *inode)
 
 		trans->block_rsv = rsv;
 
-		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
-						 &control);
+		ret = btrfs_truncate_inode_items(trans, root, &control);
 		trans->block_rsv = &fs_info->trans_block_rsv;
 		btrfs_end_transaction(trans);
 		btrfs_btree_balance_dirty(fs_info);
@@ -8534,6 +8534,7 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)
 static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 {
 	struct btrfs_truncate_control control = {
+		.inode = BTRFS_I(inode),
 		.ino = btrfs_ino(BTRFS_I(inode)),
 		.min_type = BTRFS_EXTENT_DATA_KEY,
 		.clear_extent_range = true,
@@ -8621,8 +8622,7 @@ static int btrfs_truncate(struct inode *inode, bool skip_writeback)
 					ALIGN(new_size, fs_info->sectorsize),
 					(u64)-1, 0);
 
-		ret = btrfs_truncate_inode_items(trans, root, BTRFS_I(inode),
-						 &control);
+		ret = btrfs_truncate_inode_items(trans, root, &control);
 
 		inode_sub_bytes(inode, control.sub_bytes);
 		btrfs_inode_safe_disk_i_size_write(BTRFS_I(inode),
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index fb3bf7cc21c5..80520ad1de4f 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -4104,8 +4104,7 @@ static int truncate_inode_items(struct btrfs_trans_handle *trans,
 	int ret;
 
 	do {
-		ret = btrfs_truncate_inode_items(trans, log_root, inode,
-						 &control);
+		ret = btrfs_truncate_inode_items(trans, log_root, &control);
 	} while (ret == -EAGAIN);
 
 	return ret;
-- 
2.26.3

[PATCH 14/18] btrfs: convert BUG_ON() in btrfs_truncate_inode_items to ASSERT

[Josef Bacik]

We have a correctness BUG_ON() in btrfs_truncate_inode_items to make
sure that we're always using min_type == BTRFS_EXTENT_DATA_KEY if
new_size is > 0.  Convert this to an ASSERT.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index aee374e18131..652b7069f63d 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -476,7 +476,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	bool should_throttle = false;
 
 	ASSERT(control->inode || !control->clear_extent_range);
-	BUG_ON(new_size > 0 && control->min_type != BTRFS_EXTENT_DATA_KEY);
+	ASSERT(new_size == 0 || control->min_type == BTRFS_EXTENT_DATA_KEY);
 
 	control->last_size = new_size;
 	control->sub_bytes = 0;
-- 
2.26.3

[PATCH 15/18] btrfs: convert BUG() for pending_del_nr into an ASSERT

[Josef Bacik]

This is a logic correctness check, convert it into an ASSERT() instead
of a BUG().

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 652b7069f63d..9c44cf30d930 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -655,6 +655,9 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 		else
 			control->last_size = new_size;
 		if (del_item) {
+			ASSERT(!pending_del_nr ||
+			       ((path->slots[0] + 1) == pending_del_slot));
+
 			if (!pending_del_nr) {
 				/* no pending yet, add ourselves */
 				pending_del_slot = path->slots[0];
@@ -664,8 +667,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 				/* hop on the pending chunk */
 				pending_del_nr++;
 				pending_del_slot = path->slots[0];
-			} else {
-				BUG();
 			}
 		} else {
 			break;
-- 
2.26.3

[PATCH 16/18] btrfs: combine extra if statements in btrfs_truncate_inode_items

[Josef Bacik]

We have

if (del_item)
	// do something
else
	// something else
if (del_item)
	// do yet another thing
else
	// something else entirely

back to back in btrfs_truncate_inode_items, collapse these two sets of
if statements into one.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 9c44cf30d930..257a72ec8993 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -650,14 +650,11 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			}
 		}
 
-		if (del_item)
-			control->last_size = found_key.offset;
-		else
-			control->last_size = new_size;
 		if (del_item) {
 			ASSERT(!pending_del_nr ||
 			       ((path->slots[0] + 1) == pending_del_slot));
 
+			control->last_size = found_key.offset;
 			if (!pending_del_nr) {
 				/* no pending yet, add ourselves */
 				pending_del_slot = path->slots[0];
@@ -669,6 +666,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 				pending_del_slot = path->slots[0];
 			}
 		} else {
+			control->last_size = new_size;
 			break;
 		}
 		should_throttle = false;
-- 
2.26.3

[PATCH 17/18] btrfs: make should_throttle loop local in btrfs_truncate_inode_items

[Josef Bacik]

We reset this bool on every loop through the truncate loop, make this
variable local to the loop.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/inode-item.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 257a72ec8993..4d3a6448aca2 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -473,7 +473,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 	int ret;
 	u64 bytes_deleted = 0;
 	bool be_nice = false;
-	bool should_throttle = false;
 
 	ASSERT(control->inode || !control->clear_extent_range);
 	ASSERT(new_size == 0 || control->min_type == BTRFS_EXTENT_DATA_KEY);
@@ -525,6 +524,7 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 
 	while (1) {
 		u64 clear_start = 0, clear_len = 0, extent_start = 0;
+		bool should_throttle = false;
 
 		fi = NULL;
 		leaf = path->nodes[0];
@@ -669,7 +669,6 @@ int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
 			control->last_size = new_size;
 			break;
 		}
-		should_throttle = false;
 
 		if (del_item && extent_start != 0 && !control->skip_ref_updates) {
 			struct btrfs_ref ref = { 0 };
-- 
2.26.3

[PATCH 18/18] btrfs: do not check -EAGAIN when truncating inodes in the log root

[Josef Bacik]

We only throttle the btrfs_truncate_inode_items if the root is
SHAREABLE, which isn't set on the log root, which means this loop is
unnecessary.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/tree-log.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index 80520ad1de4f..d7322ceb8aa2 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -4101,13 +4101,8 @@ static int truncate_inode_items(struct btrfs_trans_handle *trans,
 		.min_type = min_type,
 		.skip_ref_updates = true,
 	};
-	int ret;
-
-	do {
-		ret = btrfs_truncate_inode_items(trans, log_root, &control);
-	} while (ret == -EAGAIN);
 
-	return ret;
+	return btrfs_truncate_inode_items(trans, log_root, &control);
 }
 
 static void fill_inode_item(struct btrfs_trans_handle *trans,
-- 
2.26.3

Re: [PATCH 00/18] Truncate cleanups and preparation work

[Filipe Manana]

On Fri, Dec 03, 2021 at 05:18:02PM -0500, Josef Bacik wrote:
> Hello,
> 
> The first thing I'm implementing with the garbage collection tree is
> btrfs_truncate_inode_items() on evicted inodes.  However
> btrfs_truncate_inode_items() has a lot of oddities that it's grown over the
> years, and requires having a valid btrfs_inode to use.  We don't really want to
> have to look up the old inode to do the truncate, we just want to do the tree
> operaitons to delete all of the objects and extents.
> 
> Enter this patch series, I've cleaned up btrfs_truncate_inode_items(), moved as
> much of the inode operations out to the respective callers, and cleaned up the
> argument passing and such to make it a little cleaner.
> 
> We still have to pass in the inode for the ^NO_HOLES case for the normal
> truncate path, but other than that I've stripped it down so that we can pass in
> a NULL inode and get all the work done.
> 
> This has the nice side-effect of cleaning up a lot of our
> 
> if (root == LOG_ROOT)
> 	// do something else
> 
> checks in this helper, and hopefully makes it more straightforward to
> understand.  Thanks,
> 
> Josef
> 
> Josef Bacik (18):
>   btrfs: add an inode-item.h
>   btrfs: move btrfs_truncate_inode_items to inode-item.c
>   btrfs: move extent locking outside of btrfs_truncate_inode_items
>   btrfs: remove free space cache inode check in
>     btrfs_truncate_inode_items
>   btrfs: move btrfs_kill_delayed_inode_items into evict
>   btrfs: remove found_extent from btrfs_truncate_inode_items
>   btrfs: add btrfs_truncate_control struct
>   btrfs: only update i_size in truncate paths that care
>   btrfs: only call inode_sub_bytes in truncate paths that care
>   btrfs: control extent reference updates with a control flag for
>     truncate
>   btrfs: use a flag to control when to clear the file extent range
>   btrfs: pass the ino via btrfs_truncate_control
>   btrfs: add inode to btrfs_truncate_control
>   btrfs: convert BUG_ON() in btrfs_truncate_inode_items to ASSERT
>   btrfs: convert BUG() for pending_del_nr into an ASSERT
>   btrfs: combine extra if statements in btrfs_truncate_inode_items
>   btrfs: make should_throttle loop local in btrfs_truncate_inode_items
>   btrfs: do not check -EAGAIN when truncating inodes in the log root
> 
>  fs/btrfs/ctree.h            |  34 ---
>  fs/btrfs/delayed-inode.c    |   1 +
>  fs/btrfs/free-space-cache.c |  31 ++-
>  fs/btrfs/inode-item.c       | 334 ++++++++++++++++++++++++++
>  fs/btrfs/inode-item.h       |  86 +++++++
>  fs/btrfs/inode.c            | 452 +++++-------------------------------
>  fs/btrfs/relocation.c       |   1 +
>  fs/btrfs/tree-log.c         |  15 +-
>  8 files changed, 511 insertions(+), 443 deletions(-)
>  create mode 100644 fs/btrfs/inode-item.h

Looks good, and it passed one iteration of fstests here.

For the whole series:

Reviewed-by: Filipe Manana <fdmanana@suse.com>

Thanks.

> 
> -- 
> 2.26.3
> 

Re: [PATCH 01/18] btrfs: add an inode-item.h

[David Sterba]

On Fri, Dec 03, 2021 at 05:18:03PM -0500, Josef Bacik wrote:
> We have a few helpers in inode-item.c, and I'm going to make a few
> changes to how we do truncate in the future, so break out these
> definitions into their own header file to trim down ctree.h some and
> make it easier to do the work on truncate in the future.
> 
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> ---
>  fs/btrfs/ctree.h            | 30 ------------------------------
>  fs/btrfs/delayed-inode.c    |  1 +
>  fs/btrfs/free-space-cache.c |  1 +
>  fs/btrfs/inode-item.c       |  1 +
>  fs/btrfs/inode-item.h       | 37 +++++++++++++++++++++++++++++++++++++
>  fs/btrfs/inode.c            |  1 +
>  fs/btrfs/relocation.c       |  1 +
>  fs/btrfs/tree-log.c         |  1 +
>  8 files changed, 43 insertions(+), 30 deletions(-)
>  create mode 100644 fs/btrfs/inode-item.h
> 
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index dfee4b403da1..f33cae82e7dd 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
> @@ -3126,36 +3126,6 @@ int btrfs_del_orphan_item(struct btrfs_trans_handle *trans,
>  			  struct btrfs_root *root, u64 offset);
>  int btrfs_find_orphan_item(struct btrfs_root *root, u64 offset);
>  
> -/* inode-item.c */
> -int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
> -			   struct btrfs_root *root,
> -			   const char *name, int name_len,
> -			   u64 inode_objectid, u64 ref_objectid, u64 index);
> -int btrfs_del_inode_ref(struct btrfs_trans_handle *trans,
> -			   struct btrfs_root *root,
> -			   const char *name, int name_len,
> -			   u64 inode_objectid, u64 ref_objectid, u64 *index);
> -int btrfs_insert_empty_inode(struct btrfs_trans_handle *trans,
> -			     struct btrfs_root *root,
> -			     struct btrfs_path *path, u64 objectid);
> -int btrfs_lookup_inode(struct btrfs_trans_handle *trans, struct btrfs_root
> -		       *root, struct btrfs_path *path,
> -		       struct btrfs_key *location, int mod);
> -
> -struct btrfs_inode_extref *
> -btrfs_lookup_inode_extref(struct btrfs_trans_handle *trans,
> -			  struct btrfs_root *root,
> -			  struct btrfs_path *path,
> -			  const char *name, int name_len,
> -			  u64 inode_objectid, u64 ref_objectid, int ins_len,
> -			  int cow);
> -
> -struct btrfs_inode_ref *btrfs_find_name_in_backref(struct extent_buffer *leaf,
> -						   int slot, const char *name,
> -						   int name_len);
> -struct btrfs_inode_extref *btrfs_find_name_in_ext_backref(
> -		struct extent_buffer *leaf, int slot, u64 ref_objectid,
> -		const char *name, int name_len);
>  /* file-item.c */
>  struct btrfs_dio_private;
>  int btrfs_del_csums(struct btrfs_trans_handle *trans,
> diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c
> index 6f134f2c5e68..748bf6b0d860 100644
> --- a/fs/btrfs/delayed-inode.c
> +++ b/fs/btrfs/delayed-inode.c
> @@ -13,6 +13,7 @@
>  #include "ctree.h"
>  #include "qgroup.h"
>  #include "locking.h"
> +#include "inode-item.h"
>  
>  #define BTRFS_DELAYED_WRITEBACK		512
>  #define BTRFS_DELAYED_BACKGROUND	128
> diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
> index 132209ff2262..55e1be703a39 100644
> --- a/fs/btrfs/free-space-cache.c
> +++ b/fs/btrfs/free-space-cache.c
> @@ -23,6 +23,7 @@
>  #include "block-group.h"
>  #include "discard.h"
>  #include "subpage.h"
> +#include "inode-item.h"
>  
>  #define BITS_PER_BITMAP		(PAGE_SIZE * 8UL)
>  #define MAX_CACHE_BYTES_PER_GIG	SZ_64K
> diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
> index 56755ce9a907..72593a93c43c 100644
> --- a/fs/btrfs/inode-item.c
> +++ b/fs/btrfs/inode-item.c
> @@ -4,6 +4,7 @@
>   */
>  
>  #include "ctree.h"
> +#include "inode-item.h"
>  #include "disk-io.h"
>  #include "transaction.h"
>  #include "print-tree.h"
> diff --git a/fs/btrfs/inode-item.h b/fs/btrfs/inode-item.h
> new file mode 100644
> index 000000000000..cb4b140e3b7d
> --- /dev/null
> +++ b/fs/btrfs/inode-item.h
> @@ -0,0 +1,37 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef BTRFS_INODE_ITEM_H
> +#define BTRFS_INODE_ITEM_H
> +

Please don't forget to add forward declarations for all structure types
and include all other necessary headers for eg. u64 when adding a new
header. I'll add it now.

Re: [PATCH 00/18] Truncate cleanups and preparation work

[David Sterba]

On Fri, Dec 03, 2021 at 05:18:02PM -0500, Josef Bacik wrote:
> Hello,
> 
> The first thing I'm implementing with the garbage collection tree is
> btrfs_truncate_inode_items() on evicted inodes.  However
> btrfs_truncate_inode_items() has a lot of oddities that it's grown over the
> years, and requires having a valid btrfs_inode to use.  We don't really want to
> have to look up the old inode to do the truncate, we just want to do the tree
> operaitons to delete all of the objects and extents.
> 
> Enter this patch series, I've cleaned up btrfs_truncate_inode_items(), moved as
> much of the inode operations out to the respective callers, and cleaned up the
> argument passing and such to make it a little cleaner.
> 
> We still have to pass in the inode for the ^NO_HOLES case for the normal
> truncate path, but other than that I've stripped it down so that we can pass in
> a NULL inode and get all the work done.
> 
> This has the nice side-effect of cleaning up a lot of our
> 
> if (root == LOG_ROOT)
> 	// do something else
> 
> checks in this helper, and hopefully makes it more straightforward to
> understand.  Thanks,
> 
> Josef
> 
> Josef Bacik (18):
>   btrfs: add an inode-item.h
>   btrfs: move btrfs_truncate_inode_items to inode-item.c
>   btrfs: move extent locking outside of btrfs_truncate_inode_items
>   btrfs: remove free space cache inode check in
>     btrfs_truncate_inode_items
>   btrfs: move btrfs_kill_delayed_inode_items into evict
>   btrfs: remove found_extent from btrfs_truncate_inode_items
>   btrfs: add btrfs_truncate_control struct
>   btrfs: only update i_size in truncate paths that care
>   btrfs: only call inode_sub_bytes in truncate paths that care
>   btrfs: control extent reference updates with a control flag for
>     truncate
>   btrfs: use a flag to control when to clear the file extent range
>   btrfs: pass the ino via btrfs_truncate_control
>   btrfs: add inode to btrfs_truncate_control
>   btrfs: convert BUG_ON() in btrfs_truncate_inode_items to ASSERT
>   btrfs: convert BUG() for pending_del_nr into an ASSERT
>   btrfs: combine extra if statements in btrfs_truncate_inode_items
>   btrfs: make should_throttle loop local in btrfs_truncate_inode_items
>   btrfs: do not check -EAGAIN when truncating inodes in the log root

Added to misc-next, thanks.