[PATCH v3 0/3] btrfs: separate BLOCK_GROUP_TREE feature from extent-tree-v2

[PATCH v3 0/3] btrfs: separate BLOCK_GROUP_TREE feature from extent-tree-v2

[Qu Wenruo]

[Changelog]
v3:
- Add artificial requirement on fst and no-holes
  This is to address the concern from Josef on too many feature
  combinations.
  In fact I believe future features should also have hard requirement
  on the latest default features.

v2:
- Rebased to latest misc-next
  This fixes some random crash not related to btrfs.

- Fix some missing conversion due to bad branch
  I got my code messed up due to some bad local branch naming.
  The previous version sent to the ML lacks some essential conversion.

  Now it can properly pass full fstest run with block group tree.

This is the kernel part to revive block-group-tree feature.

Thanfully unlike btrfs-progs, the changes to kernel is much smaller, and
we can re-use most of the infrastructures from the extent-tree-v2
preparation patches.

But there are still some changes needed:

- Enhance unsupporter compat RO flags handling
  Extent tree is only needed for read-write opeartions, and for
  unsupported compat RO flags, we should not do any write into the fs.

  So this patch will make the kernel to skip block group items search
  if there is any unsupport RO compat flags.

  And really make the incoming block-group-tree feature compat RO.

  Unfortunately, we need that patch to be backported, or older kernels
  will still reject RO mounts of fses with block-group-tree feature.

- Don't store block group root into super block
  There is no special reason for block group root to be stored in super
  block.
  We should review those preparation patches with more scrutiny.


For the proper time reduction introduced by this patchset, the old data
should still be correct, as the on-disk format is not changed.
https://lwn.net/Articles/801990/


Qu Wenruo (3):
  btrfs: enhance unsupported compat RO flags handling
  btrfs: don't save block group root into super block
  btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2

 fs/btrfs/block-group.c     | 11 ++++++-
 fs/btrfs/block-rsv.c       |  1 +
 fs/btrfs/ctree.h           | 30 +++----------------
 fs/btrfs/disk-io.c         | 59 +++++++++++++++++++-------------------
 fs/btrfs/disk-io.h         |  2 +-
 fs/btrfs/super.c           |  9 ++++++
 fs/btrfs/sysfs.c           |  2 ++
 fs/btrfs/transaction.c     |  8 ------
 include/uapi/linux/btrfs.h |  6 ++++
 9 files changed, 62 insertions(+), 66 deletions(-)

-- 
2.37.0

[PATCH v3 1/3] btrfs: enhance unsupported compat RO flags handling

[Qu Wenruo]

Currently there are two corner cases not handling compat RO flags
correctly:

- Remount
  We can still mount the fs RO with compat RO flags, then remount it RW.
  We should not allow any write into a fs with unsupported RO flags.

- Still try to search block group items
  In fact, behavior/on-disk format change to extent tree should not
  need a full incompat flag.

  And since we can ensure fs with unsupported RO flags never got any
  writes (with above case fixed), then we can even skip block group
  items search at mount time.

This patch will enhance the unsupported RO compat flags by:

- Reject RW remount if there is unsupported RO compat flags

- Go dummy block group items directly for unsupported RO compat flags
  In fact, only changes to chunk/subvolume/root/csum trees should go
  incompat flags.

The latter part should allow future change to extent tree to be compat
RO flags.

Thus this patch also needs to be backported to all stable trees.

Cc: stable@vger.kernel.org
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/block-group.c | 11 ++++++++++-
 fs/btrfs/super.c       |  9 +++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 38a740d5297c..4a78136bcd17 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2186,7 +2186,16 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
 	int need_clear = 0;
 	u64 cache_gen;
 
-	if (!root)
+	/*
+	 * Either no extent root (with ibadroots rescue option) or we have
+	 * unsupporter RO options. The fs can never be mounted RW, so no
+	 * need to waste time search block group items.
+	 *
+	 * This also allows new extent tree related changes to be RO compat,
+	 * no need for a full incompat flag.
+	 */
+	if (!root || (btrfs_super_compat_ro_flags(info->super_copy) &
+		      ~BTRFS_FEATURE_COMPAT_RO_SUPP))
 		return fill_dummy_bgs(info);
 
 	key.objectid = 0;
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 4c7089b1681b..7d3213e67fb5 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -2110,6 +2110,15 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)
 			ret = -EINVAL;
 			goto restore;
 		}
+		if (btrfs_super_compat_ro_flags(fs_info->super_copy) &
+		    ~BTRFS_FEATURE_COMPAT_RO_SUPP) {
+			btrfs_err(fs_info,
+		"can not remount read-write due to unsupported optional flags 0x%llx",
+				btrfs_super_compat_ro_flags(fs_info->super_copy) &
+				~BTRFS_FEATURE_COMPAT_RO_SUPP);
+			ret = -EINVAL;
+			goto restore;
+		}
 		if (fs_info->fs_devices->rw_devices == 0) {
 			ret = -EACCES;
 			goto restore;
-- 
2.37.0

[PATCH v3 2/3] btrfs: don't save block group root into super block

[Qu Wenruo]

The extent tree v2 (thankfully not yet fully materialized) needs a
new root for storing all block group items.

My initial proposal years ago just added a new tree rootid, and load it
from tree root, just like what we did for quota/free space tree/uuid/extent
roots.

But the extent tree v2 patches introduced a completely new (and to me,
wasteful) way to store block group tree root into super block.

Currently there are only 3 trees stored in super blocks, and they all
have their valid reasons:

- Chunk root
  Needed for bootstrap.

- Tree root
  Really the entrance of all trees.

- Log root
  This is special as log root has to be updated out of existing
  transaction mechanism.

There is not even any reason to put block group root into super blocks,
the block group tree is updated at the same timing as old extent tree,
no need for extra bootstrap/out-of-transaction update.

So just move block group root from super block into tree root.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/block-rsv.c   |  1 +
 fs/btrfs/ctree.h       | 27 ++-------------------------
 fs/btrfs/disk-io.c     | 40 ++++++++++++++++++++--------------------
 fs/btrfs/transaction.c |  8 --------
 4 files changed, 23 insertions(+), 53 deletions(-)

diff --git a/fs/btrfs/block-rsv.c b/fs/btrfs/block-rsv.c
index 06be0644dd37..6ce704d3bdd2 100644
--- a/fs/btrfs/block-rsv.c
+++ b/fs/btrfs/block-rsv.c
@@ -424,6 +424,7 @@ void btrfs_init_root_block_rsv(struct btrfs_root *root)
 	case BTRFS_CSUM_TREE_OBJECTID:
 	case BTRFS_EXTENT_TREE_OBJECTID:
 	case BTRFS_FREE_SPACE_TREE_OBJECTID:
+	case BTRFS_BLOCK_GROUP_TREE_OBJECTID:
 		root->block_rsv = &fs_info->delayed_refs_rsv;
 		break;
 	case BTRFS_ROOT_TREE_OBJECTID:
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 51c480439263..bdcf0e24882f 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -280,14 +280,9 @@ struct btrfs_super_block {
 	/* the UUID written into btree blocks */
 	u8 metadata_uuid[BTRFS_FSID_SIZE];
 
-	/* Extent tree v2 */
-	__le64 block_group_root;
-	__le64 block_group_root_generation;
-	u8 block_group_root_level;
-
 	/* future expansion */
-	u8 reserved8[7];
-	__le64 reserved[25];
+	u8 reserved8[8];
+	__le64 reserved[27];
 	u8 sys_chunk_array[BTRFS_SYSTEM_CHUNK_ARRAY_SIZE];
 	struct btrfs_root_backup super_roots[BTRFS_NUM_BACKUP_ROOTS];
 
@@ -2480,17 +2475,6 @@ BTRFS_SETGET_STACK_FUNCS(backup_bytes_used, struct btrfs_root_backup,
 BTRFS_SETGET_STACK_FUNCS(backup_num_devices, struct btrfs_root_backup,
 		   num_devices, 64);
 
-/*
- * For extent tree v2 we overload the extent root with the block group root, as
- * we will have multiple extent roots.
- */
-BTRFS_SETGET_STACK_FUNCS(backup_block_group_root, struct btrfs_root_backup,
-			 extent_root, 64);
-BTRFS_SETGET_STACK_FUNCS(backup_block_group_root_gen, struct btrfs_root_backup,
-			 extent_root_gen, 64);
-BTRFS_SETGET_STACK_FUNCS(backup_block_group_root_level,
-			 struct btrfs_root_backup, extent_root_level, 8);
-
 /* struct btrfs_balance_item */
 BTRFS_SETGET_FUNCS(balance_flags, struct btrfs_balance_item, flags, 64);
 
@@ -2623,13 +2607,6 @@ BTRFS_SETGET_STACK_FUNCS(super_cache_generation, struct btrfs_super_block,
 BTRFS_SETGET_STACK_FUNCS(super_magic, struct btrfs_super_block, magic, 64);
 BTRFS_SETGET_STACK_FUNCS(super_uuid_tree_generation, struct btrfs_super_block,
 			 uuid_tree_generation, 64);
-BTRFS_SETGET_STACK_FUNCS(super_block_group_root, struct btrfs_super_block,
-			 block_group_root, 64);
-BTRFS_SETGET_STACK_FUNCS(super_block_group_root_generation,
-			 struct btrfs_super_block,
-			 block_group_root_generation, 64);
-BTRFS_SETGET_STACK_FUNCS(super_block_group_root_level, struct btrfs_super_block,
-			 block_group_root_level, 8);
 
 int btrfs_super_csum_size(const struct btrfs_super_block *s);
 const char *btrfs_super_csum_name(u16 csum_type);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 6268dafeeb2d..0e280f84cb15 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1526,6 +1526,9 @@ static struct btrfs_root *btrfs_get_global_root(struct btrfs_fs_info *fs_info,
 	if (objectid == BTRFS_UUID_TREE_OBJECTID)
 		return btrfs_grab_root(fs_info->uuid_root) ?
 			fs_info->uuid_root : ERR_PTR(-ENOENT);
+	if (objectid == BTRFS_BLOCK_GROUP_TREE_OBJECTID)
+		return btrfs_grab_root(fs_info->block_group_root) ?
+			fs_info->block_group_root : ERR_PTR(-ENOENT);
 	if (objectid == BTRFS_FREE_SPACE_TREE_OBJECTID) {
 		struct btrfs_root *root = btrfs_global_root(fs_info, &key);
 
@@ -1982,14 +1985,7 @@ static void backup_super_roots(struct btrfs_fs_info *info)
 	btrfs_set_backup_chunk_root_level(root_backup,
 			       btrfs_header_level(info->chunk_root->node));
 
-	if (btrfs_fs_incompat(info, EXTENT_TREE_V2)) {
-		btrfs_set_backup_block_group_root(root_backup,
-					info->block_group_root->node->start);
-		btrfs_set_backup_block_group_root_gen(root_backup,
-			btrfs_header_generation(info->block_group_root->node));
-		btrfs_set_backup_block_group_root_level(root_backup,
-			btrfs_header_level(info->block_group_root->node));
-	} else {
+	if (!btrfs_fs_incompat(info, EXTENT_TREE_V2)) {
 		struct btrfs_root *extent_root = btrfs_extent_root(info, 0);
 		struct btrfs_root *csum_root = btrfs_csum_root(info, 0);
 
@@ -2531,10 +2527,24 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info)
 	if (ret)
 		return ret;
 
-	location.objectid = BTRFS_DEV_TREE_OBJECTID;
 	location.type = BTRFS_ROOT_ITEM_KEY;
 	location.offset = 0;
 
+	if (btrfs_fs_incompat(fs_info, EXTENT_TREE_V2)) {
+		location.objectid = BTRFS_BLOCK_GROUP_TREE_OBJECTID;
+		root = btrfs_read_tree_root(tree_root, &location);
+		if (IS_ERR(root)) {
+			if (!btrfs_test_opt(fs_info, IGNOREBADROOTS)) {
+				ret = PTR_ERR(root);
+				goto out;
+			}
+		} else {
+			set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
+			fs_info->block_group_root = root;
+		}
+	}
+
+	location.objectid = BTRFS_DEV_TREE_OBJECTID;
 	root = btrfs_read_tree_root(tree_root, &location);
 	if (IS_ERR(root)) {
 		if (!btrfs_test_opt(fs_info, IGNOREBADROOTS)) {
@@ -2862,17 +2872,7 @@ static int load_important_roots(struct btrfs_fs_info *fs_info)
 		btrfs_warn(fs_info, "couldn't read tree root");
 		return ret;
 	}
-
-	if (!btrfs_fs_incompat(fs_info, EXTENT_TREE_V2))
-		return 0;
-
-	bytenr = btrfs_super_block_group_root(sb);
-	gen = btrfs_super_block_group_root_generation(sb);
-	level = btrfs_super_block_group_root_level(sb);
-	ret = load_super_root(fs_info->block_group_root, bytenr, gen, level);
-	if (ret)
-		btrfs_warn(fs_info, "couldn't read block group root");
-	return ret;
+	return 0;
 }
 
 static int __cold init_tree_roots(struct btrfs_fs_info *fs_info)
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 6e3b2cb6a04a..363e6e1d9d30 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -1935,14 +1935,6 @@ static void update_super_roots(struct btrfs_fs_info *fs_info)
 		super->cache_generation = 0;
 	if (test_bit(BTRFS_FS_UPDATE_UUID_TREE_GEN, &fs_info->flags))
 		super->uuid_tree_generation = root_item->generation;
-
-	if (btrfs_fs_incompat(fs_info, EXTENT_TREE_V2)) {
-		root_item = &fs_info->block_group_root->root_item;
-
-		super->block_group_root = root_item->bytenr;
-		super->block_group_root_generation = root_item->generation;
-		super->block_group_root_level = root_item->level;
-	}
 }
 
 int btrfs_transaction_in_commit(struct btrfs_fs_info *info)
-- 
2.37.0

[PATCH v3 3/3] btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2

[Qu Wenruo]

The problem of long mount time caused by block group item search is
already known for over 7 years, and the solution of block group tree is
proposed at least for 5 years.

There is really no need to bound this feature into extent tree v2, just
introduce compat RO flag, BLOCK_GROUP_TREE, to correctly solve the
problem.

All the code handling block group root is already in the upstream
kernel, thus this patch really only needs to introduce the new compat RO
flag.

This patch introduces one extra artificial limitation on block group
tree feature, that free space cache v2 and no-holes feature must be
enabled to use this new compat RO feature.

This artificial requirement is mostly to reduce the test combinations,
and can be a guideline for future features, to mostly rely on the latest
default features.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/ctree.h           |  3 ++-
 fs/btrfs/disk-io.c         | 23 +++++++++++------------
 fs/btrfs/disk-io.h         |  2 +-
 fs/btrfs/sysfs.c           |  2 ++
 include/uapi/linux/btrfs.h |  6 ++++++
 5 files changed, 22 insertions(+), 14 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index bdcf0e24882f..46fcbd355ccd 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -302,7 +302,8 @@ static_assert(sizeof(struct btrfs_super_block) == BTRFS_SUPER_INFO_SIZE);
 #define BTRFS_FEATURE_COMPAT_RO_SUPP			\
 	(BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE |	\
 	 BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID | \
-	 BTRFS_FEATURE_COMPAT_RO_VERITY)
+	 BTRFS_FEATURE_COMPAT_RO_VERITY |		\
+	 BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE)
 
 #define BTRFS_FEATURE_COMPAT_RO_SAFE_SET	0ULL
 #define BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR	0ULL
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 0e280f84cb15..64ec3e519f6c 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1985,7 +1985,7 @@ static void backup_super_roots(struct btrfs_fs_info *info)
 	btrfs_set_backup_chunk_root_level(root_backup,
 			       btrfs_header_level(info->chunk_root->node));
 
-	if (!btrfs_fs_incompat(info, EXTENT_TREE_V2)) {
+	if (!btrfs_fs_compat_ro(info, BLOCK_GROUP_TREE)) {
 		struct btrfs_root *extent_root = btrfs_extent_root(info, 0);
 		struct btrfs_root *csum_root = btrfs_csum_root(info, 0);
 
@@ -2530,7 +2530,7 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info)
 	location.type = BTRFS_ROOT_ITEM_KEY;
 	location.offset = 0;
 
-	if (btrfs_fs_incompat(fs_info, EXTENT_TREE_V2)) {
+	if (btrfs_fs_compat_ro(fs_info, BLOCK_GROUP_TREE)) {
 		location.objectid = BTRFS_BLOCK_GROUP_TREE_OBJECTID;
 		root = btrfs_read_tree_root(tree_root, &location);
 		if (IS_ERR(root)) {
@@ -2715,6 +2715,15 @@ static int validate_super(struct btrfs_fs_info *fs_info,
 		ret = -EINVAL;
 	}
 
+	/* Artificial requirement for block group tree to use newer features. */
+	if (btrfs_fs_compat_ro(fs_info, BLOCK_GROUP_TREE) &&
+	    (!btrfs_fs_compat_ro(fs_info, FREE_SPACE_TREE_VALID) ||
+	     !btrfs_fs_incompat(fs_info, NO_HOLES))) {
+		btrfs_err(fs_info,
+"block-group-tree feature requires space cache v2 and no-holes features");
+		ret = -EINVAL;
+	}
+
 	if (memcmp(fs_info->fs_devices->metadata_uuid, sb->dev_item.fsid,
 		   BTRFS_FSID_SIZE) != 0) {
 		btrfs_err(fs_info,
@@ -2884,16 +2893,6 @@ static int __cold init_tree_roots(struct btrfs_fs_info *fs_info)
 	int ret = 0;
 	int i;
 
-	if (btrfs_fs_incompat(fs_info, EXTENT_TREE_V2)) {
-		struct btrfs_root *root;
-
-		root = btrfs_alloc_root(fs_info, BTRFS_BLOCK_GROUP_TREE_OBJECTID,
-					GFP_KERNEL);
-		if (!root)
-			return -ENOMEM;
-		fs_info->block_group_root = root;
-	}
-
 	for (i = 0; i < BTRFS_NUM_BACKUP_ROOTS; i++) {
 		if (handle_error) {
 			if (!IS_ERR(tree_root->node))
diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h
index 47ad8e0a2d33..66fd20cd4009 100644
--- a/fs/btrfs/disk-io.h
+++ b/fs/btrfs/disk-io.h
@@ -103,7 +103,7 @@ static inline struct btrfs_root *btrfs_grab_root(struct btrfs_root *root)
 
 static inline struct btrfs_root *btrfs_block_group_root(struct btrfs_fs_info *fs_info)
 {
-	if (btrfs_fs_incompat(fs_info, EXTENT_TREE_V2))
+	if (btrfs_fs_compat_ro(fs_info, BLOCK_GROUP_TREE))
 		return fs_info->block_group_root;
 	return btrfs_extent_root(fs_info, 0);
 }
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 43d45a0cfcda..aa4d6e226d72 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -286,6 +286,7 @@ BTRFS_FEAT_ATTR_INCOMPAT(skinny_metadata, SKINNY_METADATA);
 BTRFS_FEAT_ATTR_INCOMPAT(no_holes, NO_HOLES);
 BTRFS_FEAT_ATTR_INCOMPAT(metadata_uuid, METADATA_UUID);
 BTRFS_FEAT_ATTR_COMPAT_RO(free_space_tree, FREE_SPACE_TREE);
+BTRFS_FEAT_ATTR_COMPAT_RO(block_group_tree, BLOCK_GROUP_TREE);
 BTRFS_FEAT_ATTR_INCOMPAT(raid1c34, RAID1C34);
 #ifdef CONFIG_BLK_DEV_ZONED
 BTRFS_FEAT_ATTR_INCOMPAT(zoned, ZONED);
@@ -316,6 +317,7 @@ static struct attribute *btrfs_supported_feature_attrs[] = {
 	BTRFS_FEAT_ATTR_PTR(no_holes),
 	BTRFS_FEAT_ATTR_PTR(metadata_uuid),
 	BTRFS_FEAT_ATTR_PTR(free_space_tree),
+	BTRFS_FEAT_ATTR_PTR(block_group_tree),
 	BTRFS_FEAT_ATTR_PTR(raid1c34),
 #ifdef CONFIG_BLK_DEV_ZONED
 	BTRFS_FEAT_ATTR_PTR(zoned),
diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h
index f54dc91e4025..5f79610e1c72 100644
--- a/include/uapi/linux/btrfs.h
+++ b/include/uapi/linux/btrfs.h
@@ -290,6 +290,12 @@ struct btrfs_ioctl_fs_info_args {
 #define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID	(1ULL << 1)
 #define BTRFS_FEATURE_COMPAT_RO_VERITY			(1ULL << 2)
 
+/*
+ * Put all block group items into a dedicate block group tree, greatly
+ * reduce mount time for large fs.
+ */
+#define BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE	(1ULL << 5)
+
 #define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF	(1ULL << 0)
 #define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL	(1ULL << 1)
 #define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS	(1ULL << 2)
-- 
2.37.0

Re: [PATCH v3 0/3] btrfs: separate BLOCK_GROUP_TREE feature from extent-tree-v2

[David Sterba]

On Tue, Aug 09, 2022 at 01:02:15PM +0800, Qu Wenruo wrote:
> [Changelog]
> v3:
> - Add artificial requirement on fst and no-holes
>   This is to address the concern from Josef on too many feature
>   combinations.
>   In fact I believe future features should also have hard requirement
>   on the latest default features.
> 
> v2:
> - Rebased to latest misc-next
>   This fixes some random crash not related to btrfs.
> 
> - Fix some missing conversion due to bad branch
>   I got my code messed up due to some bad local branch naming.
>   The previous version sent to the ML lacks some essential conversion.
> 
>   Now it can properly pass full fstest run with block group tree.
> 
> This is the kernel part to revive block-group-tree feature.
> 
> Thanfully unlike btrfs-progs, the changes to kernel is much smaller, and
> we can re-use most of the infrastructures from the extent-tree-v2
> preparation patches.
> 
> But there are still some changes needed:
> 
> - Enhance unsupporter compat RO flags handling
>   Extent tree is only needed for read-write opeartions, and for
>   unsupported compat RO flags, we should not do any write into the fs.
> 
>   So this patch will make the kernel to skip block group items search
>   if there is any unsupport RO compat flags.
> 
>   And really make the incoming block-group-tree feature compat RO.
> 
>   Unfortunately, we need that patch to be backported, or older kernels
>   will still reject RO mounts of fses with block-group-tree feature.
> 
> - Don't store block group root into super block
>   There is no special reason for block group root to be stored in super
>   block.
>   We should review those preparation patches with more scrutiny.
> 
> 
> For the proper time reduction introduced by this patchset, the old data
> should still be correct, as the on-disk format is not changed.
> https://lwn.net/Articles/801990/
> 
> 
> Qu Wenruo (3):
>   btrfs: enhance unsupported compat RO flags handling
>   btrfs: don't save block group root into super block
>   btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2

Adding this to for-next.

Re: [PATCH v3 1/3] btrfs: enhance unsupported compat RO flags handling

[David Sterba]

On Tue, Aug 09, 2022 at 01:02:16PM +0800, Qu Wenruo wrote:
> Currently there are two corner cases not handling compat RO flags
> correctly:
> 
> - Remount
>   We can still mount the fs RO with compat RO flags, then remount it RW.
>   We should not allow any write into a fs with unsupported RO flags.
> 
> - Still try to search block group items
>   In fact, behavior/on-disk format change to extent tree should not
>   need a full incompat flag.
> 
>   And since we can ensure fs with unsupported RO flags never got any
>   writes (with above case fixed), then we can even skip block group
>   items search at mount time.
> 
> This patch will enhance the unsupported RO compat flags by:
> 
> - Reject RW remount if there is unsupported RO compat flags
> 
> - Go dummy block group items directly for unsupported RO compat flags
>   In fact, only changes to chunk/subvolume/root/csum trees should go
>   incompat flags.
> 
> The latter part should allow future change to extent tree to be compat
> RO flags.
> 
> Thus this patch also needs to be backported to all stable trees.
> 
> Cc: stable@vger.kernel.org

This applies cleanly only to 5.19, anything else would need a separate
backport. I'm planning to send this patch among 6.0 fixes so this
should give us time to get it to older stable kernels before the block
group tree is released.

Re: [PATCH v3 3/3] btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2

[David Sterba]

On Tue, Aug 09, 2022 at 01:02:18PM +0800, Qu Wenruo wrote:
> --- a/include/uapi/linux/btrfs.h
> +++ b/include/uapi/linux/btrfs.h
> @@ -290,6 +290,12 @@ struct btrfs_ioctl_fs_info_args {
>  #define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID	(1ULL << 1)
>  #define BTRFS_FEATURE_COMPAT_RO_VERITY			(1ULL << 2)
>  
> +/*
> + * Put all block group items into a dedicate block group tree, greatly
> + * reduce mount time for large fs.
> + */
> +#define BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE	(1ULL << 5)

Is there a reason to skip the bits 3 and 4? Ie. why isn't this (1 << 3) ?

Re: [PATCH v3 1/3] btrfs: enhance unsupported compat RO flags handling

[Qu Wenruo]

On 2022/9/5 22:53, David Sterba wrote:
> On Tue, Aug 09, 2022 at 01:02:16PM +0800, Qu Wenruo wrote:
>> Currently there are two corner cases not handling compat RO flags
>> correctly:
>>
>> - Remount
>>    We can still mount the fs RO with compat RO flags, then remount it RW.
>>    We should not allow any write into a fs with unsupported RO flags.
>>
>> - Still try to search block group items
>>    In fact, behavior/on-disk format change to extent tree should not
>>    need a full incompat flag.
>>
>>    And since we can ensure fs with unsupported RO flags never got any
>>    writes (with above case fixed), then we can even skip block group
>>    items search at mount time.
>>
>> This patch will enhance the unsupported RO compat flags by:
>>
>> - Reject RW remount if there is unsupported RO compat flags
>>
>> - Go dummy block group items directly for unsupported RO compat flags
>>    In fact, only changes to chunk/subvolume/root/csum trees should go
>>    incompat flags.
>>
>> The latter part should allow future change to extent tree to be compat
>> RO flags.
>>
>> Thus this patch also needs to be backported to all stable trees.
>>
>> Cc: stable@vger.kernel.org
>
> This applies cleanly only to 5.19, anything else would need a separate
> backport. I'm planning to send this patch among 6.0 fixes so this
> should give us time to get it to older stable kernels before the block
> group tree is released.

No problem, in fact I (normally) only start backports when I got the
failed to apply message from Greg.

So I'd go the regular backport cycle for it.

Thanks,
Qu

Re: [PATCH v3 3/3] btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2

[Qu Wenruo]

On 2022/9/5 23:01, David Sterba wrote:
> On Tue, Aug 09, 2022 at 01:02:18PM +0800, Qu Wenruo wrote:
>> --- a/include/uapi/linux/btrfs.h
>> +++ b/include/uapi/linux/btrfs.h
>> @@ -290,6 +290,12 @@ struct btrfs_ioctl_fs_info_args {
>>   #define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID	(1ULL << 1)
>>   #define BTRFS_FEATURE_COMPAT_RO_VERITY			(1ULL << 2)
>>
>> +/*
>> + * Put all block group items into a dedicate block group tree, greatly
>> + * reduce mount time for large fs.
>> + */
>> +#define BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE	(1ULL << 5)
>
> Is there a reason to skip the bits 3 and 4? Ie. why isn't this (1 << 3) ?

I was saving 3 and 4 for EXTRA_SUPER_RESERVE (extra reserved space after
1MiB, needs a new member in super block) and WRITE_INTENT for raid56.

But obviously bg-tree is way easier to push, so would you mind to change
it (1<<3)?

Thanks,
Qu

Re: [PATCH v3 3/3] btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2

[David Sterba]

On Tue, Sep 06, 2022 at 06:37:50AM +0800, Qu Wenruo wrote:
> 
> 
> On 2022/9/5 23:01, David Sterba wrote:
> > On Tue, Aug 09, 2022 at 01:02:18PM +0800, Qu Wenruo wrote:
> >> --- a/include/uapi/linux/btrfs.h
> >> +++ b/include/uapi/linux/btrfs.h
> >> @@ -290,6 +290,12 @@ struct btrfs_ioctl_fs_info_args {
> >>   #define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID	(1ULL << 1)
> >>   #define BTRFS_FEATURE_COMPAT_RO_VERITY			(1ULL << 2)
> >>
> >> +/*
> >> + * Put all block group items into a dedicate block group tree, greatly
> >> + * reduce mount time for large fs.
> >> + */
> >> +#define BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE	(1ULL << 5)
> >
> > Is there a reason to skip the bits 3 and 4? Ie. why isn't this (1 << 3) ?
> 
> I was saving 3 and 4 for EXTRA_SUPER_RESERVE (extra reserved space after
> 1MiB, needs a new member in super block) and WRITE_INTENT for raid56.
> 
> But obviously bg-tree is way easier to push, so would you mind to change
> it (1<<3)?

I see, the other patchset is not even in the queue yet so 1<<3 is the
first free value, changed in git.

Re: [PATCH v3 0/3] btrfs: separate BLOCK_GROUP_TREE feature from extent-tree-v2

[David Sterba]

On Tue, Aug 23, 2022 at 10:01:06PM +0200, David Sterba wrote:
> On Tue, Aug 09, 2022 at 01:02:15PM +0800, Qu Wenruo wrote:
> > [Changelog]
> > v3:
> > - Add artificial requirement on fst and no-holes
> >   This is to address the concern from Josef on too many feature
> >   combinations.
> >   In fact I believe future features should also have hard requirement
> >   on the latest default features.
> > 
> > v2:
> > - Rebased to latest misc-next
> >   This fixes some random crash not related to btrfs.
> > 
> > - Fix some missing conversion due to bad branch
> >   I got my code messed up due to some bad local branch naming.
> >   The previous version sent to the ML lacks some essential conversion.
> > 
> >   Now it can properly pass full fstest run with block group tree.
> > 
> > This is the kernel part to revive block-group-tree feature.
> > 
> > Thanfully unlike btrfs-progs, the changes to kernel is much smaller, and
> > we can re-use most of the infrastructures from the extent-tree-v2
> > preparation patches.
> > 
> > But there are still some changes needed:
> > 
> > - Enhance unsupporter compat RO flags handling
> >   Extent tree is only needed for read-write opeartions, and for
> >   unsupported compat RO flags, we should not do any write into the fs.
> > 
> >   So this patch will make the kernel to skip block group items search
> >   if there is any unsupport RO compat flags.
> > 
> >   And really make the incoming block-group-tree feature compat RO.
> > 
> >   Unfortunately, we need that patch to be backported, or older kernels
> >   will still reject RO mounts of fses with block-group-tree feature.
> > 
> > - Don't store block group root into super block
> >   There is no special reason for block group root to be stored in super
> >   block.
> >   We should review those preparation patches with more scrutiny.
> > 
> > 
> > For the proper time reduction introduced by this patchset, the old data
> > should still be correct, as the on-disk format is not changed.
> > https://lwn.net/Articles/801990/
> > 
> > 
> > Qu Wenruo (3):
> >   btrfs: enhance unsupported compat RO flags handling
> >   btrfs: don't save block group root into super block
> >   btrfs: separate BLOCK_GROUP_TREE compat RO flag from EXTENT_TREE_V2
> 
> Adding this to for-next.

Moved to misc-next, thanks.