[PATCH v3 0/3] btrfs: Introduce new rescue= mount options

[PATCH v3 0/3] btrfs: Introduce new rescue= mount options

[Qu Wenruo]

This patchset can be fetched from github:
https://github.com/adam900710/linux/tree/rescue_options

The basis HEAD is v5.2-rc2 tag.

There are quite a lot btrfs extent tree corruption report in the mail
list.
Since btrfs will do mount time block group item search, one corrupted
leaf containing block group item will prevent the whole fs to be
mounted.

This patchset will try to address the problem by introducing a new mount
option, "rescue=skip_bg", as a last-resort rescue.
Of course this option will have a lot of restriction to prevent further
screwing up the fs, including:
- Permanent RO
  No remount rw is allowed

- No dirty log
  Either clean the log or use rescue=no_log_replay mount option

This "rescue=skip_bg" has some advantage compared to user space tool
like "btrfs-restore":
- Unified recovery tool
  User can use any tool they're familiar with, as long as the kernel
  doesn't panic.

- More info for subvolume.
  "btrfs subv list" can work now!


Also move the following mount options to "rescue=" group:
- nologreplay
  to rescue=no_log_replay

- usebackuproot
  to rescue=use_backup_root

Old options are still available for compatibility purpose, but they are
deprecated in favor of new 'rescue=' super option.

Different rescue sub options can be separated by ':', like:
"rescue=no_log_replay:skip_bg:use_backup_root".
Or the traditional but longer way like:
"rescue=no_log_replay,rescue=skip_bg"

The separation character is chosen by:
- No conflicts with existing character
  Especially no conflict with ','.

- No extra escaping/quota
  Original plan is ';', but since it'll be interpreted by bash, it's
  changed to current ':'.

Changelog:
v2:
- Introduce 'rescue=' super option.
- Rename original 'usebackuproot' and 'nologreplay'.
  It at least makes my vim spell check happier.
- Remove 'recovery' mount option.
  As its successor is now deprecated, not need to keep the predecessor.

v2.1:
- Rebase to v5.1-rc4.
- Fix the typos in the cover letter.

v3:
- Rebased to v5.2-rc2.
- Update commit message to include an example for "rescue=" options.
- Remove unnecessary exclusion of super blocks spaces and block group
  ro.
  This seems to cause incorrect df output.

Qu Wenruo (3):
  btrfs: Remove "recovery" mount option
  btrfs: Introduce "rescue=" mount option
  btrfs: Introduce new mount option to skip block group items scan

 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     | 29 +++++++++++--
 fs/btrfs/extent-tree.c | 50 ++++++++++++++++++++++
 fs/btrfs/super.c       | 96 ++++++++++++++++++++++++++++++++++++------
 fs/btrfs/volumes.c     |  7 +++
 5 files changed, 167 insertions(+), 16 deletions(-)

-- 
2.22.0

[PATCH v3 1/3] btrfs: Remove "recovery" mount option

[Qu Wenruo]

Commit 8dcddfa048de ("btrfs: Introduce new mount option usebackuproot to
replace recovery") deprecates "recovery" mount option in 2016, and it
has been 3 years, it should be OK to remove "recovery" mount option.

As we're even going to deprecate the successor, "usebackuproot" mount
option, there isn't really much need to keep the original option.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/super.c | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 0645ec428b4f..64f20725615a 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -311,7 +311,6 @@ enum {
 	Opt_defrag, Opt_nodefrag,
 	Opt_discard, Opt_nodiscard,
 	Opt_nologreplay,
-	Opt_norecovery,
 	Opt_ratio,
 	Opt_rescan_uuid_tree,
 	Opt_skip_balance,
@@ -329,7 +328,6 @@ enum {
 
 	/* Deprecated options */
 	Opt_alloc_start,
-	Opt_recovery,
 	Opt_subvolrootid,
 
 	/* Debugging options */
@@ -374,7 +372,6 @@ static const match_table_t tokens = {
 	{Opt_discard, "discard"},
 	{Opt_nodiscard, "nodiscard"},
 	{Opt_nologreplay, "nologreplay"},
-	{Opt_norecovery, "norecovery"},
 	{Opt_ratio, "metadata_ratio=%u"},
 	{Opt_rescan_uuid_tree, "rescan_uuid_tree"},
 	{Opt_skip_balance, "skip_balance"},
@@ -396,7 +393,6 @@ static const match_table_t tokens = {
 
 	/* Deprecated options */
 	{Opt_alloc_start, "alloc_start=%s"},
-	{Opt_recovery, "recovery"},
 	{Opt_subvolrootid, "subvolrootid=%d"},
 
 	/* Debugging options */
@@ -670,7 +666,6 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_clear_and_info(info, NOTREELOG,
 					     "enabling tree log");
 			break;
-		case Opt_norecovery:
 		case Opt_nologreplay:
 			btrfs_set_and_info(info, NOLOGREPLAY,
 					   "disabling log replay at mount time");
@@ -759,10 +754,6 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_clear_and_info(info, AUTO_DEFRAG,
 					     "disabling auto defrag");
 			break;
-		case Opt_recovery:
-			btrfs_warn(info,
-				   "'recovery' is deprecated, use 'usebackuproot' instead");
-			/* fall through */
 		case Opt_usebackuproot:
 			btrfs_info(info,
 				   "trying to use backup root at mount time");
-- 
2.22.0

[PATCH v3 2/3] btrfs: Introduce "rescue=" mount option

[Qu Wenruo]

This patch introduces a new "rescue=" mount option for all those mount
options for data recovery.

Different rescue sub options are seperated by ':'. E.g
"ro,rescue=no_log_replay:use_backup_root".
(The original plan is to use ';', but ';' needs to be escaped/quoted,
or it will be interpreted by bash)

The following mount options are converted to "rescue=", old mount
options are deprecated but still available for compatibility purpose:

- usebackuproot
  Now it's "rescue=use_backup_root"

- nologreplay
  Now it's "rescue=no_log_replay"

The new underscore is here to make the option more readable and make
spell check happier.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/super.c | 65 +++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 62 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 64f20725615a..4512f25dcf69 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -310,7 +310,6 @@ enum {
 	Opt_datasum, Opt_nodatasum,
 	Opt_defrag, Opt_nodefrag,
 	Opt_discard, Opt_nodiscard,
-	Opt_nologreplay,
 	Opt_ratio,
 	Opt_rescan_uuid_tree,
 	Opt_skip_balance,
@@ -323,7 +322,6 @@ enum {
 	Opt_subvolid,
 	Opt_thread_pool,
 	Opt_treelog, Opt_notreelog,
-	Opt_usebackuproot,
 	Opt_user_subvol_rm_allowed,
 
 	/* Deprecated options */
@@ -341,6 +339,8 @@ enum {
 #ifdef CONFIG_BTRFS_FS_REF_VERIFY
 	Opt_ref_verify,
 #endif
+	/* Rescue options */
+	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
 	Opt_err,
 };
 
@@ -401,6 +401,7 @@ static const match_table_t tokens = {
 	{Opt_check_integrity_print_mask, "check_int_print_mask=%u"},
 	{Opt_enospc_debug, "enospc_debug"},
 	{Opt_noenospc_debug, "noenospc_debug"},
+	{Opt_rescue, "rescue=%s"},
 #ifdef CONFIG_BTRFS_DEBUG
 	{Opt_fragment_data, "fragment=data"},
 	{Opt_fragment_metadata, "fragment=metadata"},
@@ -412,6 +413,55 @@ static const match_table_t tokens = {
 	{Opt_err, NULL},
 };
 
+static const match_table_t rescue_tokens = {
+	{Opt_usebackuproot, "use_backup_root"},
+	{Opt_nologreplay, "no_log_replay"},
+	{Opt_err, NULL},
+};
+
+static int parse_rescue_options(struct btrfs_fs_info *info, const char *options)
+{
+	char *opts;
+	char *orig;
+	char *p;
+	substring_t args[MAX_OPT_ARGS];
+	int ret = 0;
+
+	opts = kstrdup(options, GFP_KERNEL);
+	if (!opts)
+		return -ENOMEM;
+	orig = opts;
+
+	while ((p = strsep(&opts, ":")) != NULL) {
+		int token;
+
+		if (!*p)
+			continue;
+		token = match_token(p, rescue_tokens, args);
+		switch (token){
+		case Opt_usebackuproot:
+			btrfs_info(info,
+				   "trying to use backup root at mount time");
+			btrfs_set_opt(info->mount_opt, USEBACKUPROOT);
+			break;
+		case Opt_nologreplay:
+			btrfs_set_and_info(info, NOLOGREPLAY,
+					   "disabling log replay at mount time");
+			break;
+		case Opt_err:
+			btrfs_info(info, "unrecognized rescue option '%s'", p);
+			ret = -EINVAL;
+			goto out;
+		default:
+			break;
+		}
+
+	}
+out:
+	kfree(orig);
+	return ret;
+}
+
 /*
  * Regular mount options parser.  Everything that is needed only when
  * reading in a new superblock is parsed here.
@@ -667,6 +717,8 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 					     "enabling tree log");
 			break;
 		case Opt_nologreplay:
+			btrfs_warn(info,
+	"'nologreplay' is deprecated, use 'rescue=no_log_replay' instead");
 			btrfs_set_and_info(info, NOLOGREPLAY,
 					   "disabling log replay at mount time");
 			break;
@@ -755,6 +807,8 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 					     "disabling auto defrag");
 			break;
 		case Opt_usebackuproot:
+			btrfs_warn(info,
+	"'usebackuproot' is deprecated, use 'rescue=use_backup_root' instead");
 			btrfs_info(info,
 				   "trying to use backup root at mount time");
 			btrfs_set_opt(info->mount_opt, USEBACKUPROOT);
@@ -841,6 +895,11 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_set_opt(info->mount_opt, REF_VERIFY);
 			break;
 #endif
+		case Opt_rescue:
+			ret = parse_rescue_options(info, args[0].from);
+			if (ret < 0)
+				goto out;
+			break;
 		case Opt_err:
 			btrfs_info(info, "unrecognized mount option '%s'", p);
 			ret = -EINVAL;
@@ -1307,7 +1366,7 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
 	if (btrfs_test_opt(info, NOTREELOG))
 		seq_puts(seq, ",notreelog");
 	if (btrfs_test_opt(info, NOLOGREPLAY))
-		seq_puts(seq, ",nologreplay");
+		seq_puts(seq, ",rescue=no_log_replay");
 	if (btrfs_test_opt(info, FLUSHONCOMMIT))
 		seq_puts(seq, ",flushoncommit");
 	if (btrfs_test_opt(info, DISCARD))
-- 
2.22.0

[PATCH v3 3/3] btrfs: Introduce new mount option to skip block group items scan

[Qu Wenruo]

[PROBLEM]
There are some reports of corrupted fs which can't be mounted due to
corrupted extent tree.

However under such situation, it's more likely the fs/subvolume trees
are still fine.

For such case we normally go btrfs-restore and salvage as much as we
can. However btrfs-restore can't list subvolumes as "btrfs subv list",
making it harder to restore a fs.

[ENHANCEMENT]
This patch will introduce a new mount option "rescue=skip_bg" to skip
the mount time block group scan, and use chunk info purely to populate
fake block group cache.

The mount option has the following dependency:
- RO mount
  Obviously.

- No dirty log.
  Either there is no log, or use rescue=no_log_replay mount option.

- No way to remoutn RW
  Similar to rescue=no_log_replay option.

This should allow kernel to accept most extent tree corruption, and
salvage data and subvolume info.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     | 29 ++++++++++++++++++++----
 fs/btrfs/extent-tree.c | 50 ++++++++++++++++++++++++++++++++++++++++++
 fs/btrfs/super.c       | 24 +++++++++++++++++++-
 fs/btrfs/volumes.c     |  7 ++++++
 5 files changed, 106 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 0a61dff27f57..9413db8c9bf0 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1441,6 +1441,7 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info)
 #define BTRFS_MOUNT_FREE_SPACE_TREE	(1 << 26)
 #define BTRFS_MOUNT_NOLOGREPLAY		(1 << 27)
 #define BTRFS_MOUNT_REF_VERIFY		(1 << 28)
+#define BTRFS_MOUNT_SKIP_BG		(1 << 29)
 
 #define BTRFS_DEFAULT_COMMIT_INTERVAL	(30)
 #define BTRFS_DEFAULT_MAX_INLINE	(2048)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index deb74a8c191a..9701bf8ffbda 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2330,11 +2330,15 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info)
 
 	root = btrfs_read_tree_root(tree_root, &location);
 	if (IS_ERR(root)) {
-		ret = PTR_ERR(root);
-		goto out;
+		if (!btrfs_test_opt(fs_info, SKIP_BG)) {
+			ret = PTR_ERR(root);
+			goto out;
+		}
+		fs_info->extent_root = NULL;
+	} else {
+		set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
+		fs_info->extent_root = root;
 	}
-	set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
-	fs_info->extent_root = root;
 
 	location.objectid = BTRFS_DEV_TREE_OBJECTID;
 	root = btrfs_read_tree_root(tree_root, &location);
@@ -2956,6 +2960,23 @@ int open_ctree(struct super_block *sb,
 		goto fail_alloc;
 	}
 
+	/* Skip bg needs RO and no log tree replay */
+	if (btrfs_test_opt(fs_info, SKIP_BG)) {
+		if (!sb_rdonly(sb)) {
+			btrfs_err(fs_info,
+		"skip_bg mount option can only be used with read-only mount");
+			err = -EINVAL;
+			goto fail_alloc;
+		}
+		if (btrfs_super_log_root(disk_super) &&
+		    !btrfs_test_opt(fs_info, NOTREELOG)) {
+			btrfs_err(fs_info,
+	"skip_bg must be used with notreelog mount option for dirty log");
+			err = -EINVAL;
+			goto fail_alloc;
+		}
+	}
+
 	ret = btrfs_init_workqueues(fs_info, fs_devices);
 	if (ret) {
 		err = ret;
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 1aee51a9f3bf..aac9b3ce5224 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -10292,6 +10292,53 @@ static int check_chunk_block_group_mappings(struct btrfs_fs_info *fs_info)
 	return ret;
 }
 
+static int fill_dummy_bgs(struct btrfs_fs_info *fs_info)
+{
+	struct extent_map_tree *em_tree = &fs_info->mapping_tree.map_tree;
+	struct extent_map *em;
+	struct map_lookup *map;
+	struct btrfs_block_group_cache *cache;
+	struct btrfs_space_info *space_info;
+	struct rb_node *node;
+	int ret = 0;
+
+	read_lock(&em_tree->lock);
+	for (node = rb_first_cached(&em_tree->map); node;
+	     node = rb_next(node)) {
+		em = rb_entry(node, struct extent_map, rb_node);
+		map = em->map_lookup;
+		cache = btrfs_create_block_group_cache(fs_info, em->start,
+						       em->len);
+		if (!cache) {
+			ret = -ENOMEM;
+			goto out;
+		}
+
+		/* Fill dummy cache as FULL */
+		cache->flags = map->type;
+		cache->last_byte_to_unpin = (u64)-1;
+		cache->cached = BTRFS_CACHE_FINISHED;
+		btrfs_set_block_group_used(&cache->item, em->len);
+		btrfs_set_block_group_chunk_objectid(&cache->item, em->start);
+		btrfs_set_block_group_flags(&cache->item, map->type);
+		ret = btrfs_add_block_group_cache(fs_info, cache);
+		if (ret) {
+			btrfs_remove_free_space_cache(cache);
+			btrfs_put_block_group(cache);
+			goto out;
+		}
+		update_space_info(fs_info, cache->flags, em->len, em->len,
+				  0, &space_info);
+		cache->space_info = space_info;
+		link_block_group(cache);
+
+		set_avail_alloc_bits(fs_info, cache->flags);
+	}
+out:
+	read_unlock(&em_tree->lock);
+	return ret;
+}
+
 int btrfs_read_block_groups(struct btrfs_fs_info *info)
 {
 	struct btrfs_path *path;
@@ -10306,6 +10353,9 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
 	u64 feature;
 	int mixed;
 
+	if (btrfs_test_opt(info, SKIP_BG))
+		return fill_dummy_bgs(info);
+
 	feature = btrfs_super_incompat_flags(info->super_copy);
 	mixed = !!(feature & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS);
 
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 4512f25dcf69..f9b33f175e02 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -340,7 +340,7 @@ enum {
 	Opt_ref_verify,
 #endif
 	/* Rescue options */
-	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
+	Opt_rescue, Opt_usebackuproot, Opt_nologreplay, Opt_rescue_skip_bg,
 	Opt_err,
 };
 
@@ -416,6 +416,7 @@ static const match_table_t tokens = {
 static const match_table_t rescue_tokens = {
 	{Opt_usebackuproot, "use_backup_root"},
 	{Opt_nologreplay, "no_log_replay"},
+	{Opt_rescue_skip_bg, "skip_bg"},
 	{Opt_err, NULL},
 };
 
@@ -448,6 +449,10 @@ static int parse_rescue_options(struct btrfs_fs_info *info, const char *options)
 			btrfs_set_and_info(info, NOLOGREPLAY,
 					   "disabling log replay at mount time");
 			break;
+		case Opt_rescue_skip_bg:
+			btrfs_set_and_info(info, SKIP_BG,
+				"skip mount time block group searching");
+			break;
 		case Opt_err:
 			btrfs_info(info, "unrecognized rescue option '%s'", p);
 			ret = -EINVAL;
@@ -1367,6 +1372,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
 		seq_puts(seq, ",notreelog");
 	if (btrfs_test_opt(info, NOLOGREPLAY))
 		seq_puts(seq, ",rescue=no_log_replay");
+	if (btrfs_test_opt(info, SKIP_BG))
+		seq_puts(seq, ",rescue=skip_bg");
 	if (btrfs_test_opt(info, FLUSHONCOMMIT))
 		seq_puts(seq, ",flushoncommit");
 	if (btrfs_test_opt(info, DISCARD))
@@ -1796,6 +1803,14 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)
 	if (ret)
 		goto restore;
 
+	if (btrfs_test_opt(fs_info, SKIP_BG) !=
+	    (old_opts & BTRFS_MOUNT_SKIP_BG)) {
+		btrfs_err(fs_info,
+		"rescue=skip_bg mount option can't be changed during remount");
+		ret = -EINVAL;
+		goto restore;
+	}
+
 	btrfs_remount_begin(fs_info, old_opts, *flags);
 	btrfs_resize_thread_pool(fs_info,
 		fs_info->thread_pool_size, old_thread_pool_size);
@@ -1857,6 +1872,13 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)
 			goto restore;
 		}
 
+		if (btrfs_test_opt(fs_info, SKIP_BG)) {
+			btrfs_err(fs_info,
+		"remounting read-write with rescue=skip_bg is not allowed");
+			ret = -EINVAL;
+			goto restore;
+		}
+
 		ret = btrfs_cleanup_fs_roots(fs_info);
 		if (ret)
 			goto restore;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 1c2a6e4b39da..5d307b2b19d3 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -7740,6 +7740,13 @@ int btrfs_verify_dev_extents(struct btrfs_fs_info *fs_info)
 	u64 prev_dev_ext_end = 0;
 	int ret = 0;
 
+	/*
+	 * For rescue=skip_bg mount option, we're already RO and are salvaging
+	 * data, no need for such strict check.
+	 */
+	if (btrfs_test_opt(fs_info, SKIP_BG))
+		return 0;
+
 	key.objectid = 1;
 	key.type = BTRFS_DEV_EXTENT_KEY;
 	key.offset = 0;
-- 
2.22.0

Re: [PATCH v3 1/3] btrfs: Remove "recovery" mount option

[David Sterba]

On Wed, Jun 12, 2019 at 02:36:55PM +0800, Qu Wenruo wrote:
> Commit 8dcddfa048de ("btrfs: Introduce new mount option usebackuproot to
> replace recovery") deprecates "recovery" mount option in 2016, and it
> has been 3 years, it should be OK to remove "recovery" mount option.

Well, that's what we never know if it's ok o not so the deprecated
options usually stay. Eg. subvolrootid is still there.

3 years might sound a like a long time but there are old kernels that
follow the stable branches so I'd rather derive the time to removal from
that. And eg 4.4 still has the 'recovery' option not among the
deprecated, same 4.9 and 4.14.

Since 4.19 it's deprecated so we'll have to wait until that is EOL,
Dec 2020.

Re: [PATCH v3 2/3] btrfs: Introduce "rescue=" mount option

[David Sterba]

On Wed, Jun 12, 2019 at 02:36:56PM +0800, Qu Wenruo wrote:
> This patch introduces a new "rescue=" mount option for all those mount
> options for data recovery.
> 
> Different rescue sub options are seperated by ':'. E.g
> "ro,rescue=no_log_replay:use_backup_root".
> (The original plan is to use ';', but ';' needs to be escaped/quoted,
> or it will be interpreted by bash)

':' as separator is fine

> The following mount options are converted to "rescue=", old mount
> options are deprecated but still available for compatibility purpose:
> 
> - usebackuproot
>   Now it's "rescue=use_backup_root"
> 
> - nologreplay
>   Now it's "rescue=no_log_replay"
> 
> The new underscore is here to make the option more readable and make
> spell check happier.

Who uses spell checker on mount options, really? I'd expect that the new
syntax would build on top of the old so the exact same names of the
options are now shifted to the value of 'rescue='.

The usability is better with switching

  -o usebackuproot

to

  -o rescue=usebackuproot

ie. just prepending 'rescue='.

> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  fs/btrfs/super.c | 65 +++++++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 62 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index 64f20725615a..4512f25dcf69 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -310,7 +310,6 @@ enum {
>  	Opt_datasum, Opt_nodatasum,
>  	Opt_defrag, Opt_nodefrag,
>  	Opt_discard, Opt_nodiscard,
> -	Opt_nologreplay,
>  	Opt_ratio,
>  	Opt_rescan_uuid_tree,
>  	Opt_skip_balance,
> @@ -323,7 +322,6 @@ enum {
>  	Opt_subvolid,
>  	Opt_thread_pool,
>  	Opt_treelog, Opt_notreelog,
> -	Opt_usebackuproot,
>  	Opt_user_subvol_rm_allowed,
>  
>  	/* Deprecated options */
> @@ -341,6 +339,8 @@ enum {
>  #ifdef CONFIG_BTRFS_FS_REF_VERIFY
>  	Opt_ref_verify,
>  #endif
> +	/* Rescue options */
> +	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,

The options have been sorted and grouped, don't mess it up again please.
Check the list and find a better place than after the deprecated and
debugging options.

Re: [PATCH v3 3/3] btrfs: Introduce new mount option to skip block group items scan

[David Sterba]

On Wed, Jun 12, 2019 at 02:36:57PM +0800, Qu Wenruo wrote:
> [PROBLEM]
> There are some reports of corrupted fs which can't be mounted due to
> corrupted extent tree.
> 
> However under such situation, it's more likely the fs/subvolume trees
> are still fine.
> 
> For such case we normally go btrfs-restore and salvage as much as we
> can. However btrfs-restore can't list subvolumes as "btrfs subv list",
> making it harder to restore a fs.
> 
> [ENHANCEMENT]
> This patch will introduce a new mount option "rescue=skip_bg" to skip
> the mount time block group scan, and use chunk info purely to populate
> fake block group cache.
> 
> The mount option has the following dependency:
> - RO mount
>   Obviously.
> 
> - No dirty log.
>   Either there is no log, or use rescue=no_log_replay mount option.
> 
> - No way to remoutn RW
>   Similar to rescue=no_log_replay option.
> 
> This should allow kernel to accept most extent tree corruption, and
> salvage data and subvolume info.
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  fs/btrfs/ctree.h       |  1 +
>  fs/btrfs/disk-io.c     | 29 ++++++++++++++++++++----
>  fs/btrfs/extent-tree.c | 50 ++++++++++++++++++++++++++++++++++++++++++
>  fs/btrfs/super.c       | 24 +++++++++++++++++++-
>  fs/btrfs/volumes.c     |  7 ++++++
>  5 files changed, 106 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index 0a61dff27f57..9413db8c9bf0 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
> @@ -1441,6 +1441,7 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info)
>  #define BTRFS_MOUNT_FREE_SPACE_TREE	(1 << 26)
>  #define BTRFS_MOUNT_NOLOGREPLAY		(1 << 27)
>  #define BTRFS_MOUNT_REF_VERIFY		(1 << 28)
> +#define BTRFS_MOUNT_SKIP_BG		(1 << 29)
>  
>  #define BTRFS_DEFAULT_COMMIT_INTERVAL	(30)
>  #define BTRFS_DEFAULT_MAX_INLINE	(2048)
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index deb74a8c191a..9701bf8ffbda 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -2330,11 +2330,15 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info)
>  
>  	root = btrfs_read_tree_root(tree_root, &location);
>  	if (IS_ERR(root)) {
> -		ret = PTR_ERR(root);
> -		goto out;
> +		if (!btrfs_test_opt(fs_info, SKIP_BG)) {
> +			ret = PTR_ERR(root);
> +			goto out;
> +		}
> +		fs_info->extent_root = NULL;
> +	} else {
> +		set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
> +		fs_info->extent_root = root;
>  	}
> -	set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
> -	fs_info->extent_root = root;
>  
>  	location.objectid = BTRFS_DEV_TREE_OBJECTID;
>  	root = btrfs_read_tree_root(tree_root, &location);
> @@ -2956,6 +2960,23 @@ int open_ctree(struct super_block *sb,
>  		goto fail_alloc;
>  	}
>  
> +	/* Skip bg needs RO and no log tree replay */
> +	if (btrfs_test_opt(fs_info, SKIP_BG)) {
> +		if (!sb_rdonly(sb)) {
> +			btrfs_err(fs_info,
> +		"skip_bg mount option can only be used with read-only mount");
> +			err = -EINVAL;
> +			goto fail_alloc;
> +		}
> +		if (btrfs_super_log_root(disk_super) &&
> +		    !btrfs_test_opt(fs_info, NOTREELOG)) {
> +			btrfs_err(fs_info,
> +	"skip_bg must be used with notreelog mount option for dirty log");
> +			err = -EINVAL;
> +			goto fail_alloc;
> +		}
> +	}
> +
>  	ret = btrfs_init_workqueues(fs_info, fs_devices);
>  	if (ret) {
>  		err = ret;
> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
> index 1aee51a9f3bf..aac9b3ce5224 100644
> --- a/fs/btrfs/extent-tree.c
> +++ b/fs/btrfs/extent-tree.c
> @@ -10292,6 +10292,53 @@ static int check_chunk_block_group_mappings(struct btrfs_fs_info *fs_info)
>  	return ret;
>  }
>  
> +static int fill_dummy_bgs(struct btrfs_fs_info *fs_info)
> +{
> +	struct extent_map_tree *em_tree = &fs_info->mapping_tree.map_tree;
> +	struct extent_map *em;
> +	struct map_lookup *map;
> +	struct btrfs_block_group_cache *cache;
> +	struct btrfs_space_info *space_info;
> +	struct rb_node *node;
> +	int ret = 0;
> +
> +	read_lock(&em_tree->lock);
> +	for (node = rb_first_cached(&em_tree->map); node;
> +	     node = rb_next(node)) {
> +		em = rb_entry(node, struct extent_map, rb_node);
> +		map = em->map_lookup;
> +		cache = btrfs_create_block_group_cache(fs_info, em->start,
> +						       em->len);
> +		if (!cache) {
> +			ret = -ENOMEM;
> +			goto out;
> +		}
> +
> +		/* Fill dummy cache as FULL */
> +		cache->flags = map->type;
> +		cache->last_byte_to_unpin = (u64)-1;
> +		cache->cached = BTRFS_CACHE_FINISHED;
> +		btrfs_set_block_group_used(&cache->item, em->len);
> +		btrfs_set_block_group_chunk_objectid(&cache->item, em->start);
> +		btrfs_set_block_group_flags(&cache->item, map->type);
> +		ret = btrfs_add_block_group_cache(fs_info, cache);
> +		if (ret) {
> +			btrfs_remove_free_space_cache(cache);
> +			btrfs_put_block_group(cache);
> +			goto out;
> +		}
> +		update_space_info(fs_info, cache->flags, em->len, em->len,
> +				  0, &space_info);
> +		cache->space_info = space_info;
> +		link_block_group(cache);
> +
> +		set_avail_alloc_bits(fs_info, cache->flags);
> +	}
> +out:
> +	read_unlock(&em_tree->lock);
> +	return ret;
> +}
> +
>  int btrfs_read_block_groups(struct btrfs_fs_info *info)
>  {
>  	struct btrfs_path *path;
> @@ -10306,6 +10353,9 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
>  	u64 feature;
>  	int mixed;
>  
> +	if (btrfs_test_opt(info, SKIP_BG))
> +		return fill_dummy_bgs(info);
> +
>  	feature = btrfs_super_incompat_flags(info->super_copy);
>  	mixed = !!(feature & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS);
>  
> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
> index 4512f25dcf69..f9b33f175e02 100644
> --- a/fs/btrfs/super.c
> +++ b/fs/btrfs/super.c
> @@ -340,7 +340,7 @@ enum {
>  	Opt_ref_verify,
>  #endif
>  	/* Rescue options */
> -	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
> +	Opt_rescue, Opt_usebackuproot, Opt_nologreplay, Opt_rescue_skip_bg,
>  	Opt_err,
>  };
>  
> @@ -416,6 +416,7 @@ static const match_table_t tokens = {
>  static const match_table_t rescue_tokens = {
>  	{Opt_usebackuproot, "use_backup_root"},
>  	{Opt_nologreplay, "no_log_replay"},
> +	{Opt_rescue_skip_bg, "skip_bg"},
>  	{Opt_err, NULL},
>  };
>  
> @@ -448,6 +449,10 @@ static int parse_rescue_options(struct btrfs_fs_info *info, const char *options)
>  			btrfs_set_and_info(info, NOLOGREPLAY,
>  					   "disabling log replay at mount time");
>  			break;
> +		case Opt_rescue_skip_bg:
> +			btrfs_set_and_info(info, SKIP_BG,
> +				"skip mount time block group searching");
> +			break;
>  		case Opt_err:
>  			btrfs_info(info, "unrecognized rescue option '%s'", p);
>  			ret = -EINVAL;
> @@ -1367,6 +1372,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
>  		seq_puts(seq, ",notreelog");
>  	if (btrfs_test_opt(info, NOLOGREPLAY))
>  		seq_puts(seq, ",rescue=no_log_replay");
> +	if (btrfs_test_opt(info, SKIP_BG))
> +		seq_puts(seq, ",rescue=skip_bg");

Now that there are more possible values to print here, they need to
follow the format with the separator.

Re: [PATCH v3 1/3] btrfs: Remove "recovery" mount option

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 885 bytes --]



On 2019/6/12 下午10:58, David Sterba wrote:
> On Wed, Jun 12, 2019 at 02:36:55PM +0800, Qu Wenruo wrote:
>> Commit 8dcddfa048de ("btrfs: Introduce new mount option usebackuproot to
>> replace recovery") deprecates "recovery" mount option in 2016, and it
>> has been 3 years, it should be OK to remove "recovery" mount option.
> 
> Well, that's what we never know if it's ok o not so the deprecated
> options usually stay. Eg. subvolrootid is still there.
> 
> 3 years might sound a like a long time but there are old kernels that
> follow the stable branches so I'd rather derive the time to removal from
> that. And eg 4.4 still has the 'recovery' option not among the
> deprecated, same 4.9 and 4.14.
> 
> Since 4.19 it's deprecated so we'll have to wait until that is EOL,
> Dec 2020.
> 
OK, I'll remove the patch from the patchset for now.

Thanks,
Qu


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v3 2/3] btrfs: Introduce "rescue=" mount option

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 2673 bytes --]



On 2019/6/12 下午11:09, David Sterba wrote:
> On Wed, Jun 12, 2019 at 02:36:56PM +0800, Qu Wenruo wrote:
>> This patch introduces a new "rescue=" mount option for all those mount
>> options for data recovery.
>>
>> Different rescue sub options are seperated by ':'. E.g
>> "ro,rescue=no_log_replay:use_backup_root".
>> (The original plan is to use ';', but ';' needs to be escaped/quoted,
>> or it will be interpreted by bash)
> 
> ':' as separator is fine
> 
>> The following mount options are converted to "rescue=", old mount
>> options are deprecated but still available for compatibility purpose:
>>
>> - usebackuproot
>>   Now it's "rescue=use_backup_root"
>>
>> - nologreplay
>>   Now it's "rescue=no_log_replay"
>>
>> The new underscore is here to make the option more readable and make
>> spell check happier.
> 
> Who uses spell checker on mount options, really? I'd expect that the new
> syntax would build on top of the old so the exact same names of the
> options are now shifted to the value of 'rescue='.
> 
> The usability is better with switching
> 
>   -o usebackuproot
> 
> to
> 
>   -o rescue=usebackuproot

The problem is, every time I see things like usebackuproot or
nologreplay, it's not that easy to understand them just by a quick glance.
Further more, they are already rescue options, not something we would
need to use in a daily basis.

A little longer but easier to read looks valid to me in such use case.

Thanks,
Qu

> 
> ie. just prepending 'rescue='.
> 
>> Signed-off-by: Qu Wenruo <wqu@suse.com>
>> ---
>>  fs/btrfs/super.c | 65 +++++++++++++++++++++++++++++++++++++++++++++---
>>  1 file changed, 62 insertions(+), 3 deletions(-)
>>
>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>> index 64f20725615a..4512f25dcf69 100644
>> --- a/fs/btrfs/super.c
>> +++ b/fs/btrfs/super.c
>> @@ -310,7 +310,6 @@ enum {
>>  	Opt_datasum, Opt_nodatasum,
>>  	Opt_defrag, Opt_nodefrag,
>>  	Opt_discard, Opt_nodiscard,
>> -	Opt_nologreplay,
>>  	Opt_ratio,
>>  	Opt_rescan_uuid_tree,
>>  	Opt_skip_balance,
>> @@ -323,7 +322,6 @@ enum {
>>  	Opt_subvolid,
>>  	Opt_thread_pool,
>>  	Opt_treelog, Opt_notreelog,
>> -	Opt_usebackuproot,
>>  	Opt_user_subvol_rm_allowed,
>>  
>>  	/* Deprecated options */
>> @@ -341,6 +339,8 @@ enum {
>>  #ifdef CONFIG_BTRFS_FS_REF_VERIFY
>>  	Opt_ref_verify,
>>  #endif
>> +	/* Rescue options */
>> +	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
> 
> The options have been sorted and grouped, don't mess it up again please.
> Check the list and find a better place than after the deprecated and
> debugging options.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v3 2/3] btrfs: Introduce "rescue=" mount option

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3021 bytes --]



On 2019/6/13 上午9:30, Qu Wenruo wrote:
> 
> 
> On 2019/6/12 下午11:09, David Sterba wrote:
>> On Wed, Jun 12, 2019 at 02:36:56PM +0800, Qu Wenruo wrote:
>>> This patch introduces a new "rescue=" mount option for all those mount
>>> options for data recovery.
>>>
>>> Different rescue sub options are seperated by ':'. E.g
>>> "ro,rescue=no_log_replay:use_backup_root".
>>> (The original plan is to use ';', but ';' needs to be escaped/quoted,
>>> or it will be interpreted by bash)
>>
>> ':' as separator is fine
>>
>>> The following mount options are converted to "rescue=", old mount
>>> options are deprecated but still available for compatibility purpose:
>>>
>>> - usebackuproot
>>>   Now it's "rescue=use_backup_root"
>>>
>>> - nologreplay
>>>   Now it's "rescue=no_log_replay"
>>>
>>> The new underscore is here to make the option more readable and make
>>> spell check happier.
>>
>> Who uses spell checker on mount options, really? I'd expect that the new
>> syntax would build on top of the old so the exact same names of the
>> options are now shifted to the value of 'rescue='.
>>
>> The usability is better with switching
>>
>>   -o usebackuproot
>>
>> to
>>
>>   -o rescue=usebackuproot
> 
> The problem is, every time I see things like usebackuproot or
> nologreplay, it's not that easy to understand them just by a quick glance.
> Further more, they are already rescue options, not something we would
> need to use in a daily basis.
> 
> A little longer but easier to read looks valid to me in such use case.

Gentle ping here.

Do we still need to follow the hard to read mount options?
If so, I can go ahead with old names, but I really want to know if
others are OK with the existing naming.

Thanks,
Qu
> 
> Thanks,
> Qu
> 
>>
>> ie. just prepending 'rescue='.
>>
>>> Signed-off-by: Qu Wenruo <wqu@suse.com>
>>> ---
>>>  fs/btrfs/super.c | 65 +++++++++++++++++++++++++++++++++++++++++++++---
>>>  1 file changed, 62 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
>>> index 64f20725615a..4512f25dcf69 100644
>>> --- a/fs/btrfs/super.c
>>> +++ b/fs/btrfs/super.c
>>> @@ -310,7 +310,6 @@ enum {
>>>  	Opt_datasum, Opt_nodatasum,
>>>  	Opt_defrag, Opt_nodefrag,
>>>  	Opt_discard, Opt_nodiscard,
>>> -	Opt_nologreplay,
>>>  	Opt_ratio,
>>>  	Opt_rescan_uuid_tree,
>>>  	Opt_skip_balance,
>>> @@ -323,7 +322,6 @@ enum {
>>>  	Opt_subvolid,
>>>  	Opt_thread_pool,
>>>  	Opt_treelog, Opt_notreelog,
>>> -	Opt_usebackuproot,
>>>  	Opt_user_subvol_rm_allowed,
>>>  
>>>  	/* Deprecated options */
>>> @@ -341,6 +339,8 @@ enum {
>>>  #ifdef CONFIG_BTRFS_FS_REF_VERIFY
>>>  	Opt_ref_verify,
>>>  #endif
>>> +	/* Rescue options */
>>> +	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
>>
>> The options have been sorted and grouped, don't mess it up again please.
>> Check the list and find a better place than after the deprecated and
>> debugging options.
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]