All of lore.kernel.org
 help / color / mirror / Atom feed
From: Su Yue <suy.fnst@cn.fujitsu.com>
To: <linux-btrfs@vger.kernel.org>
Cc: <quwenruo.btrfs@gmx.com>, <suy.fnst@cn.fujitsu.com>
Subject: [PATCH v2 07/17] btrfs-progs: lowmem check: introduce try_avoid_extents_overwrite()
Date: Wed, 20 Dec 2017 12:57:21 +0800	[thread overview]
Message-ID: <20171220045731.19343-8-suy.fnst@cn.fujitsu.com> (raw)
In-Reply-To: <20171220045731.19343-1-suy.fnst@cn.fujitsu.com>

Define a global enum extents_operation to record extents are pinned,
excluded or new chunk is allocated for extents.
Although global variable is not so graceful, it simplifies codes much.

New function try_avoid_extents_overwrite() will try to mark block
groups full and allocate a new chunk. If it failed because of no space
or wrong used bytes(fsck-tests/004), then try to exclude metadata
blocks.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 cmds-check.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 101 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 311c8a9f45e8..9042bab93785 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -84,6 +84,15 @@ enum btrfs_check_mode {
 
 static enum btrfs_check_mode check_mode = CHECK_MODE_DEFAULT;
 
+enum lowmem_extents_operation {
+	EXTENTS_PIN,
+	EXTENTS_EXCLUDE,
+	EXTENTS_MARK_BG_FULL,
+	EXTENTS_NONE
+};
+
+static enum lowmem_extents_operation extents_operation = EXTENTS_NONE;
+
 struct extent_backref {
 	struct rb_node node;
 	unsigned int is_data:1;
@@ -13517,6 +13526,98 @@ out:
 	return err;
 }
 
+static void cleanup_excluded_extents(struct btrfs_fs_info *fs_info);
+static int end_avoid_extents_overwrite(struct btrfs_fs_info *fs_info)
+{
+	int ret;
+
+	switch (extents_operation) {
+	case EXTENTS_PIN:
+		ret = btrfs_finish_extent_commit(NULL, fs_info->extent_root,
+						 &fs_info->pinned_extents);
+		break;
+	case EXTENTS_EXCLUDE:
+		cleanup_excluded_extents(fs_info);
+		ret = 0;
+		break;
+	case EXTENTS_MARK_BG_FULL:
+		ret = modify_block_groups_cache(fs_info,
+						BTRFS_BLOCK_GROUP_METADATA, 0);
+		break;
+	case EXTENTS_NONE:
+		ret = 0;
+		break;
+	default:
+		ret = -EINVAL;
+	}
+
+	if (!ret)
+		extents_operation = EXTENTS_NONE;
+	return ret;
+}
+
+static int pin_metadata_blocks(struct btrfs_fs_info *fs_info);
+static int exclude_metadata_blocks(struct btrfs_fs_info *fs_info);
+/*
+ * NOTE: Do not call this function during transaction.
+ */
+static int avoid_extents_overwrite(struct btrfs_fs_info *fs_info,
+				   enum lowmem_extents_operation op)
+{
+	int ret;
+
+	if (op == extents_operation)
+		return 0;
+
+	switch (op) {
+	case EXTENTS_PIN:
+		ret = pin_metadata_blocks(fs_info);
+		break;
+	case EXTENTS_EXCLUDE:
+		ret = exclude_metadata_blocks(fs_info);
+		break;
+	case EXTENTS_MARK_BG_FULL:
+		ret = force_cow_in_new_chunk(fs_info);
+		break;
+	case EXTENTS_NONE:
+		ret = 0;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	/* extents_operation should be assigned anyway for latter clean up. */
+	extents_operation = op;
+
+	if (ret)
+		end_avoid_extents_overwrite(fs_info);
+	return ret;
+}
+
+static int try_avoid_extents_overwrite(struct btrfs_fs_info *fs_info)
+{
+	int ret;
+	int mixed = btrfs_fs_incompat(fs_info, MIXED_GROUPS);
+
+	if (extents_operation != EXTENTS_NONE)
+		return 0;
+	ret = avoid_extents_overwrite(fs_info, EXTENTS_MARK_BG_FULL);
+
+	/*
+	 * If there is no space left to allocate, try to exclude all metadata
+	 * blocks. Mix filesystem is unsupported.
+	 */
+	if (ret && ret == -ENOSPC && !mixed) {
+		printf(
+	"Try to exclude all metadata blcoks and extents, it may be slow\n");
+		ret = avoid_extents_overwrite(fs_info, EXTENTS_EXCLUDE);
+	}
+
+	if (ret)
+		error("failed to avoid extents overwrite %s", strerror(-ret));
+	return ret;
+}
+
 /*
  * Low memory usage version check_chunks_and_extents.
  */
-- 
2.15.1




  parent reply	other threads:[~2017-12-20  4:53 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-12-20  4:57 [PATCH v2 00/17] btrfs-progs: lowmem check: avoid extents overwrite Su Yue
2017-12-20  4:57 ` [PATCH v2 01/17] btrfs-progs: lowmem check: release path in repair_extent_data_item() Su Yue
2017-12-20  4:57 ` [PATCH v2 02/17] btrfs-progs: lowmem check: record returned errors after walk_down_tree_v2() Su Yue
2017-12-29 11:17   ` Nikolay Borisov
2018-01-02  1:44     ` Su Yue
2018-01-02  1:50     ` Su Yue
2017-12-20  4:57 ` [PATCH v2 03/17] btrfs-progs: lowmem check: assign @parent early in repair_extent_data_item() Su Yue
2017-12-20  4:57 ` [PATCH v2 04/17] btrfs-progs: lowmem check: exclude extents of metadata blocks Su Yue
2017-12-20  4:57 ` [PATCH v2 05/17] btrfs-progs: lowmem check: introduce modify_block_groups_cache() Su Yue
2017-12-20  5:38   ` Qu Wenruo
2017-12-20  4:57 ` [PATCH v2 06/17] btrfs-progs: lowmem check: introduce force_cow_in_new_chunk() Su Yue
2017-12-20  5:41   ` Qu Wenruo
2017-12-20  8:21     ` Su Yue
2017-12-20  8:37       ` Qu Wenruo
2017-12-21  6:51         ` Qu Wenruo
2017-12-21  7:06           ` Su Yue
2017-12-21  7:09           ` Su Yue
2017-12-21  7:12             ` Qu Wenruo
2017-12-26  8:17               ` Su Yue
2017-12-26 10:28                 ` Qu Wenruo
2017-12-27  1:11                   ` Su Yue
2018-01-09  7:44                     ` Qu Wenruo
2017-12-20  4:57 ` Su Yue [this message]
2017-12-20  5:46   ` [PATCH v2 07/17] btrfs-progs: lowmem check: introduce try_avoid_extents_overwrite() Qu Wenruo
2017-12-20  4:57 ` [PATCH v2 08/17] btrfs-progs: lowmem check: exclude extents if init-extent-tree in lowmem Su Yue
2017-12-20  4:57 ` [PATCH v2 09/17] btrfs-progs: lowmem check: start to remove parameters @trans " Su Yue
2017-12-20  5:51   ` Qu Wenruo
2017-12-20  4:57 ` [PATCH v2 10/17] btrfs-progs: lowmem check: remove parameter @trans of delete_extent_item() Su Yue
2017-12-20  4:57 ` [PATCH v2 11/17] btrfs-progs: lowmem check: remove parameter @trans of repair_chunk_item() Su Yue
2017-12-20  4:57 ` [PATCH v2 12/17] btrfs-progs: lowmem check: remove parameter @trans of repair_extent_item() Su Yue
2017-12-20  4:57 ` [PATCH v2 13/17] btrfs-progs: lowmem check: remove parameter @trans of check_leaf_items() Su Yue
2017-12-20  4:57 ` [PATCH v2 14/17] btrfs-progs: lowmem check: remove parameter @trans of repair_tree_back_ref() Su Yue
2017-12-20  4:57 ` [PATCH v2 15/17] btrfs-progs: lowmem check: remove parameter @trans of check_btrfs_root() Su Yue
2017-12-20  4:57 ` [PATCH v2 16/17] btrfs-progs: lowmem check: introduce repair_block_accounting() Su Yue
2017-12-20  4:57 ` [PATCH v2 17/17] btrfs-progs: lowmem check: end of removing parameters @trans in lowmem Su Yue
2017-12-20  5:59 ` [PATCH v2 00/17] btrfs-progs: lowmem check: avoid extents overwrite Qu Wenruo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171220045731.19343-8-suy.fnst@cn.fujitsu.com \
    --to=suy.fnst@cn.fujitsu.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=quwenruo.btrfs@gmx.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.