[PATCH] btrfs: add test for free space tree remounts

[PATCH] btrfs: add test for free space tree remounts

[Boris Burkov]

btrfs/131 covers a solid variety of free space tree scenarios, but it
does not cover remount scenarios. We are adding remount support for read
only btrfs filesystems to move to the free space tree, so add a few test
cases covering that workflow as well. Refactor out some common free
space tree code from btrfs/131 into common/btrfs.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 common/btrfs        | 12 +++++++++
 tests/btrfs/131     | 29 +++++++-------------
 tests/btrfs/219     | 65 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/219.out |  9 +++++++
 tests/btrfs/group   |  1 +
 5 files changed, 96 insertions(+), 20 deletions(-)
 create mode 100755 tests/btrfs/219
 create mode 100644 tests/btrfs/219.out

diff --git a/common/btrfs b/common/btrfs
index 6d452d4d..9517c0a4 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -411,3 +411,15 @@ _btrfs_forget_or_module_reload()
 
 	_reload_fs_module "btrfs"
 }
+
+# print whether or not the free space tree is enabled on the scratch dev
+_btrfs_free_space_tree_enabled()
+{
+	compat_ro="$($BTRFS_UTIL_PROG inspect-internal dump-super "$SCRATCH_DEV" | \
+		     sed -rn 's/^compat_ro_flags\s+(.*)$/\1/p')"
+	if ((compat_ro & 0x1)); then
+		echo "free space tree is enabled"
+	else
+		echo "free space tree is disabled"
+			fi
+}
diff --git a/tests/btrfs/131 b/tests/btrfs/131
index 3de7eef8..93ff59f4 100755
--- a/tests/btrfs/131
+++ b/tests/btrfs/131
@@ -52,17 +52,6 @@ mkfs_v2()
 	_scratch_unmount
 }
 
-check_fst_compat()
-{
-	compat_ro="$($BTRFS_UTIL_PROG inspect-internal dump-super "$SCRATCH_DEV" | \
-		     sed -rn 's/^compat_ro_flags\s+(.*)$/\1/p')"
-	if ((compat_ro & 0x1)); then
-		echo "free space tree is enabled"
-	else
-		echo "free space tree is disabled"
-	fi
-}
-
 # Mount options might interfere.
 export MOUNT_OPTIONS=""
 
@@ -76,19 +65,19 @@ export MOUNT_OPTIONS=""
 mkfs_v1
 echo "Using free space cache"
 _scratch_mount -o space_cache=v1
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v1
 echo "Enabling free space tree"
 _scratch_mount -o space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v1
 echo "Disabling free space cache and enabling free space tree"
 _scratch_mount -o clear_cache,space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 # When the free space tree is enabled:
@@ -106,32 +95,32 @@ _try_scratch_mount -o space_cache=v1 >/dev/null 2>&1 || echo "mount failed"
 mkfs_v2
 echo "Mounting existing free space tree"
 _scratch_mount
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 _scratch_mount -o space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v2
 echo "Recreating free space tree"
 _scratch_mount -o clear_cache,space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 mkfs_v2
 _scratch_mount -o clear_cache
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v2
 echo "Disabling free space tree"
 _scratch_mount -o clear_cache,nospace_cache
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v2
 echo "Reverting to free space cache"
 _scratch_mount -o clear_cache,space_cache=v1
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 # success, all done
diff --git a/tests/btrfs/219 b/tests/btrfs/219
new file mode 100755
index 00000000..1b889b78
--- /dev/null
+++ b/tests/btrfs/219
@@ -0,0 +1,65 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Facebook.  All Rights Reserved.
+#
+# FS QA Test 219
+#
+# Test free space tree remount scenarios.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_btrfs_command inspect-internal dump-super
+_require_btrfs_fs_feature free_space_tree
+
+# Remount:
+# -o space_cache=v1; -o remount,space_cache=v2: error
+# -o space_cache=v1,ro; -o remount,space_cache=v2: success
+echo "Trying to remount with free space tree"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount -o clear_cache,space_cache=v1
+_btrfs_free_space_tree_enabled
+_try_scratch_mount -o remount,space_cache=v2 >/dev/null 2>&1 || echo "remount failed"
+_scratch_unmount
+
+echo "Remount read only fs with free space tree"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount -o clear_cache,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_mount -o remount,ro,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_mount -o remount,space_cache=v2
+_btrfs_free_space_tree_enabled
+# ensure the free space tree is sticky across reboot
+_scratch_unmount
+_scratch_mount
+_btrfs_free_space_tree_enabled
+_scratch_unmount
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/219.out b/tests/btrfs/219.out
new file mode 100644
index 00000000..0b22e258
--- /dev/null
+++ b/tests/btrfs/219.out
@@ -0,0 +1,9 @@
+QA output created by 219
+Trying to remount with free space tree
+free space tree is disabled
+remount failed
+Remount read only fs with free space tree
+free space tree is disabled
+free space tree is disabled
+free space tree is enabled
+free space tree is enabled
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 3295856d..f4dbfafb 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -221,3 +221,4 @@
 216 auto quick seed
 217 auto quick trim dangerous
 218 auto quick volume
+219 auto quick
-- 
2.24.1

Re: [PATCH] btrfs: add test for free space tree remounts

[Nikolay Borisov]

On 1.09.20 г. 2:05 ч., Boris Burkov wrote:
> btrfs/131 covers a solid variety of free space tree scenarios, but it
> does not cover remount scenarios. We are adding remount support for read
> only btrfs filesystems to move to the free space tree, so add a few test
> cases covering that workflow as well. Refactor out some common free
> space tree code from btrfs/131 into common/btrfs.
> 
> Signed-off-by: Boris Burkov <boris@bur.io>

Overall looks good, for a couple of minor cleanup suggestions see below.
Still:

Reviewed-by: Nikolay Borisov <nborisov@suse.com>

> ---
>  common/btrfs        | 12 +++++++++
>  tests/btrfs/131     | 29 +++++++-------------
>  tests/btrfs/219     | 65 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/219.out |  9 +++++++
>  tests/btrfs/group   |  1 +
>  5 files changed, 96 insertions(+), 20 deletions(-)
>  create mode 100755 tests/btrfs/219
>  create mode 100644 tests/btrfs/219.out
> 
> diff --git a/common/btrfs b/common/btrfs
> index 6d452d4d..9517c0a4 100644
> --- a/common/btrfs
> +++ b/common/btrfs

<snip>

> diff --git a/tests/btrfs/131 b/tests/btrfs/131
> index 3de7eef8..93ff59f4 100755
> --- a/tests/btrfs/131
> +++ b/tests/btrfs/131
> @@ -52,17 +52,6 @@ mkfs_v2()
>  	_scratch_unmount

<snip>


>  # success, all done
> diff --git a/tests/btrfs/219 b/tests/btrfs/219
> new file mode 100755
> index 00000000..1b889b78
> --- /dev/null
> +++ b/tests/btrfs/219
> @@ -0,0 +1,65 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Facebook.  All Rights Reserved.
> +#
> +# FS QA Test 219
> +#
> +# Test free space tree remount scenarios.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"

<snip>
\
> +# Remount:
> +# -o space_cache=v1; -o remount,space_cache=v2: error
> +# -o space_cache=v1,ro; -o remount,space_cache=v2: success
> +echo "Trying to remount with free space tree"
> +_scratch_mkfs >/dev/null 2>&1
> +_scratch_mount -o clear_cache,space_cache=v1
> +_btrfs_free_space_tree_enabled
> +_try_scratch_mount -o remount,space_cache=v2 >/dev/null 2>&1 || echo "remount failed"

nit: _scratch_remount space_cache=v2 || echo "remount failed"

it will save you typing "-o remount" ;D

> +_scratch_unmount
> +
> +echo "Remount read only fs with free space tree"
> +_scratch_mkfs >/dev/null 2>&1
> +_scratch_mount -o clear_cache,space_cache=v1
> +_btrfs_free_space_tree_enabled
> +_scratch_mount -o remount,ro,space_cache=v1

ditto

> +_btrfs_free_space_tree_enabled
> +_scratch_mount -o remount,space_cache=v2

ditto

> +_btrfs_free_space_tree_enabled
> +# ensure the free space tree is sticky across reboot
> +_scratch_unmount
> +_scratch_mount

_scratch_cycle_mount

> +_btrfs_free_space_tree_enabled
> +_scratch_unmount
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/219.out b/tests/btrfs/219.out
> new file mode 100644
> index 00000000..0b22e258
> --- /dev/null
> +++ b/tests/btrfs/219.out
> @@ -0,0 +1,9 @@
> +QA output created by 219
> +Trying to remount with free space tree
> +free space tree is disabled
> +remount failed
> +Remount read only fs with free space tree
> +free space tree is disabled
> +free space tree is disabled
> +free space tree is enabled
> +free space tree is enabled
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 3295856d..f4dbfafb 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -221,3 +221,4 @@
>  216 auto quick seed
>  217 auto quick trim dangerous
>  218 auto quick volume
> +219 auto quick
> 

[PATCH v2] btrfs: add test for free space tree remounts

[Boris Burkov]

btrfs/131 covers a solid variety of free space tree scenarios, but it
does not cover remount scenarios. We are adding remount support for read
only btrfs filesystems to move to the free space tree, so add a few test
cases covering that workflow as well. Refactor out some common free
space tree code from btrfs/131 into common/btrfs.

Signed-off-by: Boris Burkov <boris@bur.io>
---
v2:
- added a new test for ro->ro remount
- used _scratch_remount
- used _scratch_cycle_mount

 common/btrfs        | 12 ++++++++
 tests/btrfs/131     | 29 ++++++------------
 tests/btrfs/219     | 74 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/219.out | 13 ++++++++
 tests/btrfs/group   |  1 +
 5 files changed, 109 insertions(+), 20 deletions(-)
 create mode 100755 tests/btrfs/219
 create mode 100644 tests/btrfs/219.out

diff --git a/common/btrfs b/common/btrfs
index 6d452d4d..9517c0a4 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -411,3 +411,15 @@ _btrfs_forget_or_module_reload()
 
 	_reload_fs_module "btrfs"
 }
+
+# print whether or not the free space tree is enabled on the scratch dev
+_btrfs_free_space_tree_enabled()
+{
+	compat_ro="$($BTRFS_UTIL_PROG inspect-internal dump-super "$SCRATCH_DEV" | \
+		     sed -rn 's/^compat_ro_flags\s+(.*)$/\1/p')"
+	if ((compat_ro & 0x1)); then
+		echo "free space tree is enabled"
+	else
+		echo "free space tree is disabled"
+			fi
+}
diff --git a/tests/btrfs/131 b/tests/btrfs/131
index 3de7eef8..93ff59f4 100755
--- a/tests/btrfs/131
+++ b/tests/btrfs/131
@@ -52,17 +52,6 @@ mkfs_v2()
 	_scratch_unmount
 }
 
-check_fst_compat()
-{
-	compat_ro="$($BTRFS_UTIL_PROG inspect-internal dump-super "$SCRATCH_DEV" | \
-		     sed -rn 's/^compat_ro_flags\s+(.*)$/\1/p')"
-	if ((compat_ro & 0x1)); then
-		echo "free space tree is enabled"
-	else
-		echo "free space tree is disabled"
-	fi
-}
-
 # Mount options might interfere.
 export MOUNT_OPTIONS=""
 
@@ -76,19 +65,19 @@ export MOUNT_OPTIONS=""
 mkfs_v1
 echo "Using free space cache"
 _scratch_mount -o space_cache=v1
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v1
 echo "Enabling free space tree"
 _scratch_mount -o space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v1
 echo "Disabling free space cache and enabling free space tree"
 _scratch_mount -o clear_cache,space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 # When the free space tree is enabled:
@@ -106,32 +95,32 @@ _try_scratch_mount -o space_cache=v1 >/dev/null 2>&1 || echo "mount failed"
 mkfs_v2
 echo "Mounting existing free space tree"
 _scratch_mount
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 _scratch_mount -o space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v2
 echo "Recreating free space tree"
 _scratch_mount -o clear_cache,space_cache=v2
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 mkfs_v2
 _scratch_mount -o clear_cache
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v2
 echo "Disabling free space tree"
 _scratch_mount -o clear_cache,nospace_cache
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 mkfs_v2
 echo "Reverting to free space cache"
 _scratch_mount -o clear_cache,space_cache=v1
-check_fst_compat
+_btrfs_free_space_tree_enabled
 _scratch_unmount
 
 # success, all done
diff --git a/tests/btrfs/219 b/tests/btrfs/219
new file mode 100755
index 00000000..e6c8cb60
--- /dev/null
+++ b/tests/btrfs/219
@@ -0,0 +1,74 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Facebook.  All Rights Reserved.
+#
+# FS QA Test 219
+#
+# Test free space tree remount scenarios.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_btrfs_command inspect-internal dump-super
+_require_btrfs_fs_feature free_space_tree
+
+# Remount:
+# -o space_cache=v1; -o remount,space_cache=v2: error
+# -o space_cache=v1,ro; -o remount,ro,space_cache=v2: error
+# -o space_cache=v1,ro; -o remount,space_cache=v2: success
+echo "Trying to remount with free space tree"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount -o clear_cache,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_remount space_cache=v2 >/dev/null 2>&1 || echo "remount failed"
+_scratch_unmount
+
+echo "Trying to remount read-only with free space tree"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount -o clear_cache,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_remount ro,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_remount ro,space_cache=v2 >/dev/null 2>&1 || echo "remount failed"
+_scratch_unmount
+
+echo "Remount read-only to read-write with free space tree"
+_scratch_mkfs >/dev/null 2>&1
+_scratch_mount -o clear_cache,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_remount ro,space_cache=v1
+_btrfs_free_space_tree_enabled
+_scratch_remount space_cache=v2
+_btrfs_free_space_tree_enabled
+# ensure the free space tree is sticky across reboot
+_scratch_cycle_mount
+_btrfs_free_space_tree_enabled
+_scratch_unmount
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/219.out b/tests/btrfs/219.out
new file mode 100644
index 00000000..1d5624a7
--- /dev/null
+++ b/tests/btrfs/219.out
@@ -0,0 +1,13 @@
+QA output created by 219
+Trying to remount with free space tree
+free space tree is disabled
+remount failed
+Trying to remount read-only with free space tree
+free space tree is disabled
+free space tree is disabled
+remount failed
+Remount read-only to read-write with free space tree
+free space tree is disabled
+free space tree is disabled
+free space tree is enabled
+free space tree is enabled
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 3295856d..f4dbfafb 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -221,3 +221,4 @@
 216 auto quick seed
 217 auto quick trim dangerous
 218 auto quick volume
+219 auto quick
-- 
2.24.1

Re: [PATCH v2] btrfs: add test for free space tree remounts

[Nikolay Borisov]

On 4.09.20 г. 3:25 ч., Boris Burkov wrote:
> btrfs/131 covers a solid variety of free space tree scenarios, but it
> does not cover remount scenarios. We are adding remount support for read
> only btrfs filesystems to move to the free space tree, so add a few test
> cases covering that workflow as well. Refactor out some common free
> space tree code from btrfs/131 into common/btrfs.
> 
> Signed-off-by: Boris Burkov <boris@bur.io>

Generally LGTM, but I wonder if it would be beneficial to refer the
series which is supposed to make this test pass. If you look at other
patches in xfstest that's done in the header of the test i.e "This test
freespace remount behavior introduced by xxxx" and you refer to the name
of the patches (or alternatively the commit id if the patch has landed
upstream).

In any case:


Reviewed-by: Nikolay Borisov <nborisov@suse.com>

Re: [PATCH v2] btrfs: add test for free space tree remounts

[Eryu Guan]

On Fri, Sep 04, 2020 at 10:51:36AM +0300, Nikolay Borisov wrote:
> 
> 
> On 4.09.20 г. 3:25 ч., Boris Burkov wrote:
> > btrfs/131 covers a solid variety of free space tree scenarios, but it
> > does not cover remount scenarios. We are adding remount support for read
> > only btrfs filesystems to move to the free space tree, so add a few test
> > cases covering that workflow as well. Refactor out some common free
> > space tree code from btrfs/131 into common/btrfs.
> > 
> > Signed-off-by: Boris Burkov <boris@bur.io>
> 
> Generally LGTM, but I wonder if it would be beneficial to refer the
> series which is supposed to make this test pass. If you look at other
> patches in xfstest that's done in the header of the test i.e "This test
> freespace remount behavior introduced by xxxx" and you refer to the name
> of the patches (or alternatively the commit id if the patch has landed
> upstream).

Yes, mentioning the commit id or patch subject in either commit log or
test description that makes the test pass is preferred.

> 
> In any case:
> 
> 
> Reviewed-by: Nikolay Borisov <nborisov@suse.com>

Thanks for the review!

Eryu