* [RFC PATCH v3 0/3] xfs: testcases for shrinking free space in the last AG
@ 2021-03-15 11:19 Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper Gao Xiang
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Gao Xiang @ 2021-03-15 11:19 UTC (permalink / raw)
To: linux-xfs, fstests; +Cc: Darrick J. Wong, Zorro Lang, Gao Xiang
Hi,
This version matches
kernel v8: https://lore.kernel.org/r/20210305025703.3069469-1-hsiangkao@redhat.com
xfsprogs RFC: https://lore.kernel.org/r/20201028114010.545331-1-hsiangkao@redhat.com
and mainly addresses comments for the previous version.
Thanks,
Gao Xinag
changes since RFC v2:
- [1/3] add _require_scratch to _require_xfs_shrink() (Zorro);
- [1/3] drop unneeded _scratch_unmount (Zorro);
- [2/3] add _repair_scratch_fs for each test (Zorro);
- [2/3] check new size after shrinking (Zorro);
- [2/3] drop unneeded _scratch_unmount (Zorro);
- [3/3] update group as "auto growfs ioctl prealloc stress" to follow xfs/104 (Zorro);
- [3/3] use generic falloc instead of resvsp (Darrick).
Gao Xiang:
common/xfs: add a _require_xfs_shrink helper
xfs: basic functionality test for shrinking free space in the last AG
xfs: stress test for shrinking free space in the last AG
common/xfs | 11 +++++
tests/xfs/990 | 70 ++++++++++++++++++++++++++
tests/xfs/990.out | 12 +++++
tests/xfs/991 | 122 ++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/991.out | 8 +++
tests/xfs/group | 2 +
6 files changed, 225 insertions(+)
create mode 100755 tests/xfs/990
create mode 100644 tests/xfs/990.out
create mode 100755 tests/xfs/991
create mode 100644 tests/xfs/991.out
--
2.27.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper
2021-03-15 11:19 [RFC PATCH v3 0/3] xfs: testcases for shrinking free space in the last AG Gao Xiang
@ 2021-03-15 11:19 ` Gao Xiang
2021-03-28 16:18 ` Eryu Guan
2021-03-15 11:19 ` [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 3/3] xfs: stress " Gao Xiang
2 siblings, 1 reply; 12+ messages in thread
From: Gao Xiang @ 2021-03-15 11:19 UTC (permalink / raw)
To: linux-xfs, fstests; +Cc: Darrick J. Wong, Zorro Lang, Gao Xiang
In order to detect whether the current kernel supports XFS shrinking.
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
common/xfs | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/common/xfs b/common/xfs
index 2156749d..ea3b6cab 100644
--- a/common/xfs
+++ b/common/xfs
@@ -432,6 +432,17 @@ _supports_xfs_scrub()
return 0
}
+_require_xfs_shrink()
+{
+ _require_scratch
+
+ _scratch_mkfs_xfs > /dev/null
+ _scratch_mount
+ $XFS_GROWFS_PROG -D1 "$SCRATCH_MNT" 2>&1 | grep -q 'Invalid argument' || \
+ _notrun "kernel does not support shrinking"
+ _scratch_unmount
+}
+
# run xfs_check and friends on a FS.
_check_xfs_filesystem()
{
--
2.27.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG
2021-03-15 11:19 [RFC PATCH v3 0/3] xfs: testcases for shrinking free space in the last AG Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper Gao Xiang
@ 2021-03-15 11:19 ` Gao Xiang
2021-03-28 16:32 ` Eryu Guan
2021-03-15 11:19 ` [RFC PATCH v3 3/3] xfs: stress " Gao Xiang
2 siblings, 1 reply; 12+ messages in thread
From: Gao Xiang @ 2021-03-15 11:19 UTC (permalink / raw)
To: linux-xfs, fstests; +Cc: Darrick J. Wong, Zorro Lang, Gao Xiang
Add basic test to make sure the functionality works as expected.
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
tests/xfs/990 | 70 +++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/990.out | 12 ++++++++
tests/xfs/group | 1 +
3 files changed, 83 insertions(+)
create mode 100755 tests/xfs/990
create mode 100644 tests/xfs/990.out
diff --git a/tests/xfs/990 b/tests/xfs/990
new file mode 100755
index 00000000..3c79186e
--- /dev/null
+++ b/tests/xfs/990
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test 990
+#
+# XFS shrinkfs basic functionality test
+#
+# This test attempts to shrink with a small size (512K), half AG size and
+# an out-of-bound size (agsize + 1) to observe if it works as expected.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+test_shrink()
+{
+ $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT >> $seqres.full 2>&1
+ ret=$?
+
+ _scratch_unmount
+ _repair_scratch_fs >> $seqres.full
+ _scratch_mount >> $seqres.full
+
+ $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
+ . $tmp.growfs
+ [ $ret -eq 0 -a $1 -eq $dblocks ]
+}
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_shrink
+
+rm -f $seqres.full
+echo "Format and mount"
+_scratch_mkfs -dsize="$((512 * 1024 * 1024))" -dagcount=3 2>&1 | \
+ tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
+. $tmp.mkfs
+t_dblocks=$dblocks
+_scratch_mount >> $seqres.full
+
+echo "Shrink fs (small size)"
+test_shrink $((t_dblocks-512*1024/dbsize)) || \
+ _fail "Shrink fs (small size) failure"
+
+echo "Shrink fs (half AG)"
+test_shrink $((t_dblocks-agsize/2)) || \
+ _fail "Shrink fs (half AG) failure"
+
+echo "Shrink fs (out-of-bound)"
+test_shrink $((t_dblocks-agsize-1)) && \
+ _fail "Shrink fs (out-of-bound) failure"
+[ $dblocks -ne $((t_dblocks-agsize/2)) ] && \
+ _fail "dblocks changed after shrinking failure"
+
+$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full
+echo "*** done"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/990.out b/tests/xfs/990.out
new file mode 100644
index 00000000..812f89ef
--- /dev/null
+++ b/tests/xfs/990.out
@@ -0,0 +1,12 @@
+QA output created by 990
+Format and mount
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+Shrink fs (small size)
+Shrink fs (half AG)
+Shrink fs (out-of-bound)
+*** done
diff --git a/tests/xfs/group b/tests/xfs/group
index e861cec9..a7981b67 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -525,3 +525,4 @@
525 auto quick mkfs
526 auto quick mkfs
527 auto quick quota
+990 auto quick growfs
--
2.27.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [RFC PATCH v3 3/3] xfs: stress test for shrinking free space in the last AG
2021-03-15 11:19 [RFC PATCH v3 0/3] xfs: testcases for shrinking free space in the last AG Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG Gao Xiang
@ 2021-03-15 11:19 ` Gao Xiang
2021-03-28 16:46 ` Eryu Guan
2 siblings, 1 reply; 12+ messages in thread
From: Gao Xiang @ 2021-03-15 11:19 UTC (permalink / raw)
To: linux-xfs, fstests; +Cc: Darrick J. Wong, Zorro Lang, Gao Xiang
This adds a stress testcase to shrink free space as much as
possible in the last AG with background fsstress workload.
The expectation is that no crash happens with expected output.
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
tests/xfs/991 | 122 ++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/991.out | 8 +++
tests/xfs/group | 1 +
3 files changed, 131 insertions(+)
create mode 100755 tests/xfs/991
create mode 100644 tests/xfs/991.out
diff --git a/tests/xfs/991 b/tests/xfs/991
new file mode 100755
index 00000000..7e7d318e
--- /dev/null
+++ b/tests/xfs/991
@@ -0,0 +1,122 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020-2021 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test 991
+#
+# XFS online shrinkfs stress test
+#
+# This test attempts to shrink unused space as much as possible with
+# background fsstress workload. It will decrease the shrink size if
+# larger size fails. And totally repeat 2 * TIME_FACTOR times.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+create_scratch()
+{
+ _scratch_mkfs_xfs $@ | tee -a $seqres.full | \
+ _filter_mkfs 2>$tmp.mkfs >/dev/null
+ . $tmp.mkfs
+
+ if ! _try_scratch_mount 2>/dev/null; then
+ echo "failed to mount $SCRATCH_DEV"
+ exit 1
+ fi
+
+ # fix the reserve block pool to a known size so that the enospc
+ # calculations work out correctly.
+ _scratch_resvblks 1024 > /dev/null 2>&1
+}
+
+fill_scratch()
+{
+ $XFS_IO_PROG -f -c "falloc -k 0 $1" $SCRATCH_MNT/resvfile
+}
+
+stress_scratch()
+{
+ procs=3
+ nops=$((1000 * LOAD_FACTOR))
+ # -w ensures that the only ops are ones which cause write I/O
+ FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -w -p $procs \
+ -n $nops $FSSTRESS_AVOID`
+ $FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full 2>&1 &
+}
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_shrink
+_require_xfs_io_command "falloc"
+
+rm -f $seqres.full
+_scratch_mkfs_xfs | tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
+. $tmp.mkfs # extract blocksize and data size for scratch device
+
+decsize=`expr 42 \* 1048576` # shrink in chunks of this size at most
+endsize=`expr 125 \* 1048576` # stop after shrinking this big
+[ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
+
+nags=2
+totalcount=$((2 * TIME_FACTOR))
+
+while [ $totalcount -gt 0 ]; do
+ size=`expr 1010 \* 1048576` # 1010 megabytes initially
+ logblks=$(_scratch_find_xfs_min_logblocks -dsize=${size} -dagcount=${nags})
+
+ create_scratch -lsize=${logblks}b -dsize=${size} -dagcount=${nags}
+
+ for i in `seq 125 -1 90`; do
+ fillsize=`expr $i \* 1048576`
+ out="$(fill_scratch $fillsize 2>&1)"
+ echo "$out" | grep -q 'No space left on device' && continue
+ test -n "${out}" && echo "$out"
+ break
+ done
+
+ while [ $size -gt $endsize ]; do
+ stress_scratch
+ sleep 1
+
+ decb=`expr $decsize / $dbsize` # in data blocks
+ while [ $decb -gt 0 ]; do
+ sizeb=`expr $size / $dbsize - $decb`
+
+ $XFS_GROWFS_PROG -D ${sizeb} $SCRATCH_MNT \
+ >> $seqres.full 2>&1 && break
+
+ [ $decb -gt 100 ] && decb=`expr $decb + $RANDOM % 10`
+ decb=`expr $decb / 2`
+ done
+
+ wait
+ [ $decb -eq 0 ] && break
+
+ # get latest dblocks
+ $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
+ . $tmp.growfs
+
+ size=`expr $dblocks \* $dbsize`
+ _scratch_unmount
+ _repair_scratch_fs >> $seqres.full
+ _scratch_mount
+ done
+
+ _scratch_unmount
+ _repair_scratch_fs >> $seqres.full
+ totalcount=`expr $totalcount - 1`
+done
+
+echo "*** done"
+status=0
+exit
diff --git a/tests/xfs/991.out b/tests/xfs/991.out
new file mode 100644
index 00000000..e8209672
--- /dev/null
+++ b/tests/xfs/991.out
@@ -0,0 +1,8 @@
+QA output created by 991
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+*** done
diff --git a/tests/xfs/group b/tests/xfs/group
index a7981b67..cf190b59 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -526,3 +526,4 @@
526 auto quick mkfs
527 auto quick quota
990 auto quick growfs
+991 auto growfs ioctl prealloc stress
--
2.27.0
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper
2021-03-15 11:19 ` [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper Gao Xiang
@ 2021-03-28 16:18 ` Eryu Guan
2021-03-28 19:51 ` Gao Xiang
0 siblings, 1 reply; 12+ messages in thread
From: Eryu Guan @ 2021-03-28 16:18 UTC (permalink / raw)
To: Gao Xiang; +Cc: linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 15, 2021 at 07:19:24PM +0800, Gao Xiang wrote:
> In order to detect whether the current kernel supports XFS shrinking.
>
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> ---
> common/xfs | 11 +++++++++++
> 1 file changed, 11 insertions(+)
>
> diff --git a/common/xfs b/common/xfs
> index 2156749d..ea3b6cab 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -432,6 +432,17 @@ _supports_xfs_scrub()
> return 0
> }
>
> +_require_xfs_shrink()
> +{
> + _require_scratch
_require_command "$XFS_GROWFS_PROG" xfs_growfs
> +
> + _scratch_mkfs_xfs > /dev/null
> + _scratch_mount
> + $XFS_GROWFS_PROG -D1 "$SCRATCH_MNT" 2>&1 | grep -q 'Invalid argument' || \
> + _notrun "kernel does not support shrinking"
Better to describe the behavior here to explain why EINVAL means kernel
supports shrink.
Thanks,
Eryu
> + _scratch_unmount
> +}
> +
> # run xfs_check and friends on a FS.
> _check_xfs_filesystem()
> {
> --
> 2.27.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG
2021-03-15 11:19 ` [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG Gao Xiang
@ 2021-03-28 16:32 ` Eryu Guan
2021-03-28 20:06 ` Gao Xiang
0 siblings, 1 reply; 12+ messages in thread
From: Eryu Guan @ 2021-03-28 16:32 UTC (permalink / raw)
To: Gao Xiang; +Cc: linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 15, 2021 at 07:19:25PM +0800, Gao Xiang wrote:
> Add basic test to make sure the functionality works as expected.
>
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> ---
> tests/xfs/990 | 70 +++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/990.out | 12 ++++++++
> tests/xfs/group | 1 +
> 3 files changed, 83 insertions(+)
> create mode 100755 tests/xfs/990
> create mode 100644 tests/xfs/990.out
>
> diff --git a/tests/xfs/990 b/tests/xfs/990
> new file mode 100755
> index 00000000..3c79186e
> --- /dev/null
> +++ b/tests/xfs/990
> @@ -0,0 +1,70 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Red Hat, Inc. All Rights Reserved.
> +#
> +# FS QA Test 990
> +#
> +# XFS shrinkfs basic functionality test
> +#
> +# This test attempts to shrink with a small size (512K), half AG size and
> +# an out-of-bound size (agsize + 1) to observe if it works as expected.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +test_shrink()
> +{
> + $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT >> $seqres.full 2>&1
> + ret=$?
I'm not sure what's the output of xfs_growfs when shrinking filesystem,
if it's easy to do filter, but it'd be good if we could just dump the
output and let .out file match & check the test result.
> +
> + _scratch_unmount
> + _repair_scratch_fs >> $seqres.full
_repair_scratch_fs will fix corruption if there's any, and always return 0 if
completes without problems. Is _check_scratch_fs() what you want?
> + _scratch_mount >> $seqres.full
> +
> + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
> + . $tmp.growfs
> + [ $ret -eq 0 -a $1 -eq $dblocks ]
Just dump the expected size after shrink if possible.
> +}
> +
> +# real QA test starts here
> +_supported_fs xfs
Still missing _require_scratch
> +_require_xfs_shrink
> +
> +rm -f $seqres.full
> +echo "Format and mount"
> +_scratch_mkfs -dsize="$((512 * 1024 * 1024))" -dagcount=3 2>&1 | \
> + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
Better to explain why mkfs with agcount=3
> +. $tmp.mkfs
> +t_dblocks=$dblocks
> +_scratch_mount >> $seqres.full
> +
> +echo "Shrink fs (small size)"
> +test_shrink $((t_dblocks-512*1024/dbsize)) || \
> + _fail "Shrink fs (small size) failure"
If it's possible to turn test_shrink to .out file matching way, _fail is
not needed and below.
> +
> +echo "Shrink fs (half AG)"
> +test_shrink $((t_dblocks-agsize/2)) || \
> + _fail "Shrink fs (half AG) failure"
> +
> +echo "Shrink fs (out-of-bound)"
> +test_shrink $((t_dblocks-agsize-1)) && \
> + _fail "Shrink fs (out-of-bound) failure"
> +[ $dblocks -ne $((t_dblocks-agsize/2)) ] && \
> + _fail "dblocks changed after shrinking failure"
> +
> +$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full
> +echo "*** done"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/990.out b/tests/xfs/990.out
> new file mode 100644
> index 00000000..812f89ef
> --- /dev/null
> +++ b/tests/xfs/990.out
> @@ -0,0 +1,12 @@
> +QA output created by 990
> +Format and mount
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +Shrink fs (small size)
> +Shrink fs (half AG)
> +Shrink fs (out-of-bound)
> +*** done
> diff --git a/tests/xfs/group b/tests/xfs/group
> index e861cec9..a7981b67 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -525,3 +525,4 @@
> 525 auto quick mkfs
> 526 auto quick mkfs
> 527 auto quick quota
> +990 auto quick growfs
Maybe it's time to add a new 'shrinkfs' group?
Thanks,
Eryu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 3/3] xfs: stress test for shrinking free space in the last AG
2021-03-15 11:19 ` [RFC PATCH v3 3/3] xfs: stress " Gao Xiang
@ 2021-03-28 16:46 ` Eryu Guan
2021-03-28 20:09 ` Gao Xiang
0 siblings, 1 reply; 12+ messages in thread
From: Eryu Guan @ 2021-03-28 16:46 UTC (permalink / raw)
To: Gao Xiang; +Cc: linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 15, 2021 at 07:19:26PM +0800, Gao Xiang wrote:
> This adds a stress testcase to shrink free space as much as
> possible in the last AG with background fsstress workload.
>
> The expectation is that no crash happens with expected output.
>
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> ---
> tests/xfs/991 | 122 ++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/991.out | 8 +++
> tests/xfs/group | 1 +
> 3 files changed, 131 insertions(+)
> create mode 100755 tests/xfs/991
> create mode 100644 tests/xfs/991.out
>
> diff --git a/tests/xfs/991 b/tests/xfs/991
> new file mode 100755
> index 00000000..7e7d318e
> --- /dev/null
> +++ b/tests/xfs/991
> @@ -0,0 +1,122 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020-2021 Red Hat, Inc. All Rights Reserved.
> +#
> +# FS QA Test 991
> +#
> +# XFS online shrinkfs stress test
> +#
> +# This test attempts to shrink unused space as much as possible with
> +# background fsstress workload. It will decrease the shrink size if
> +# larger size fails. And totally repeat 2 * TIME_FACTOR times.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +create_scratch()
> +{
> + _scratch_mkfs_xfs $@ | tee -a $seqres.full | \
> + _filter_mkfs 2>$tmp.mkfs >/dev/null
> + . $tmp.mkfs
> +
> + if ! _try_scratch_mount 2>/dev/null; then
> + echo "failed to mount $SCRATCH_DEV"
> + exit 1
> + fi
_scratch_mount will do the check and _fail the test on mount failure.
> +
> + # fix the reserve block pool to a known size so that the enospc
> + # calculations work out correctly.
> + _scratch_resvblks 1024 > /dev/null 2>&1
> +}
> +
> +fill_scratch()
> +{
> + $XFS_IO_PROG -f -c "falloc -k 0 $1" $SCRATCH_MNT/resvfile
> +}
> +
> +stress_scratch()
> +{
> + procs=3
> + nops=$((1000 * LOAD_FACTOR))
Declare procs and nops as local.
> + # -w ensures that the only ops are ones which cause write I/O
> + FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -w -p $procs \
> + -n $nops $FSSTRESS_AVOID`
> + $FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full 2>&1 &
I think it's more explicit to run run stress_scratch in background,
instead run fsstress in background implicit.
> +}
> +
> +# real QA test starts here
> +_supported_fs xfs
_require_scratch
> +_require_xfs_shrink
> +_require_xfs_io_command "falloc"
> +
> +rm -f $seqres.full
> +_scratch_mkfs_xfs | tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
> +. $tmp.mkfs # extract blocksize and data size for scratch device
> +
> +decsize=`expr 42 \* 1048576` # shrink in chunks of this size at most
> +endsize=`expr 125 \* 1048576` # stop after shrinking this big
> +[ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
> +
> +nags=2
> +totalcount=$((2 * TIME_FACTOR))
> +
> +while [ $totalcount -gt 0 ]; do
> + size=`expr 1010 \* 1048576` # 1010 megabytes initially
> + logblks=$(_scratch_find_xfs_min_logblocks -dsize=${size} -dagcount=${nags})
> +
> + create_scratch -lsize=${logblks}b -dsize=${size} -dagcount=${nags}
> +
> + for i in `seq 125 -1 90`; do
> + fillsize=`expr $i \* 1048576`
> + out="$(fill_scratch $fillsize 2>&1)"
> + echo "$out" | grep -q 'No space left on device' && continue
> + test -n "${out}" && echo "$out"
> + break
> + done
> +
> + while [ $size -gt $endsize ]; do
> + stress_scratch
So just call
stress_scratch &
here? So it's clear that we put stress_scratch in background, and the
'wait' below is waiting for it.
Thanks,
Eryu
> + sleep 1
> +
> + decb=`expr $decsize / $dbsize` # in data blocks
> + while [ $decb -gt 0 ]; do
> + sizeb=`expr $size / $dbsize - $decb`
> +
> + $XFS_GROWFS_PROG -D ${sizeb} $SCRATCH_MNT \
> + >> $seqres.full 2>&1 && break
> +
> + [ $decb -gt 100 ] && decb=`expr $decb + $RANDOM % 10`
> + decb=`expr $decb / 2`
> + done
> +
> + wait
> + [ $decb -eq 0 ] && break
> +
> + # get latest dblocks
> + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
> + . $tmp.growfs
> +
> + size=`expr $dblocks \* $dbsize`
> + _scratch_unmount
> + _repair_scratch_fs >> $seqres.full
> + _scratch_mount
> + done
> +
> + _scratch_unmount
> + _repair_scratch_fs >> $seqres.full
> + totalcount=`expr $totalcount - 1`
> +done
> +
> +echo "*** done"
> +status=0
> +exit
> diff --git a/tests/xfs/991.out b/tests/xfs/991.out
> new file mode 100644
> index 00000000..e8209672
> --- /dev/null
> +++ b/tests/xfs/991.out
> @@ -0,0 +1,8 @@
> +QA output created by 991
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +*** done
> diff --git a/tests/xfs/group b/tests/xfs/group
> index a7981b67..cf190b59 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -526,3 +526,4 @@
> 526 auto quick mkfs
> 527 auto quick quota
> 990 auto quick growfs
> +991 auto growfs ioctl prealloc stress
> --
> 2.27.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper
2021-03-28 16:18 ` Eryu Guan
@ 2021-03-28 19:51 ` Gao Xiang
0 siblings, 0 replies; 12+ messages in thread
From: Gao Xiang @ 2021-03-28 19:51 UTC (permalink / raw)
To: Eryu Guan; +Cc: linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 29, 2021 at 12:18:25AM +0800, Eryu Guan wrote:
> On Mon, Mar 15, 2021 at 07:19:24PM +0800, Gao Xiang wrote:
> > In order to detect whether the current kernel supports XFS shrinking.
> >
> > Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> > ---
> > common/xfs | 11 +++++++++++
> > 1 file changed, 11 insertions(+)
> >
> > diff --git a/common/xfs b/common/xfs
> > index 2156749d..ea3b6cab 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -432,6 +432,17 @@ _supports_xfs_scrub()
> > return 0
> > }
> >
> > +_require_xfs_shrink()
> > +{
> > + _require_scratch
>
> _require_command "$XFS_GROWFS_PROG" xfs_growfs
will update.
>
> > +
> > + _scratch_mkfs_xfs > /dev/null
> > + _scratch_mount
> > + $XFS_GROWFS_PROG -D1 "$SCRATCH_MNT" 2>&1 | grep -q 'Invalid argument' || \
> > + _notrun "kernel does not support shrinking"
>
> Better to describe the behavior here to explain why EINVAL means kernel
> supports shrink.
hmmm.. I also don't think this check is correct now. Maybe try to shrink
one fs-block would be better. Will update later.
Thanks,
Gao Xiang
>
> Thanks,
> Eryu
>
> > + _scratch_unmount
> > +}
> > +
> > # run xfs_check and friends on a FS.
> > _check_xfs_filesystem()
> > {
> > --
> > 2.27.0
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG
2021-03-28 16:32 ` Eryu Guan
@ 2021-03-28 20:06 ` Gao Xiang
2021-03-29 2:14 ` Eryu Guan
0 siblings, 1 reply; 12+ messages in thread
From: Gao Xiang @ 2021-03-28 20:06 UTC (permalink / raw)
To: Eryu Guan; +Cc: linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 29, 2021 at 12:32:39AM +0800, Eryu Guan wrote:
> On Mon, Mar 15, 2021 at 07:19:25PM +0800, Gao Xiang wrote:
> > Add basic test to make sure the functionality works as expected.
> >
> > Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> > ---
> > tests/xfs/990 | 70 +++++++++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/990.out | 12 ++++++++
> > tests/xfs/group | 1 +
> > 3 files changed, 83 insertions(+)
> > create mode 100755 tests/xfs/990
> > create mode 100644 tests/xfs/990.out
> >
> > diff --git a/tests/xfs/990 b/tests/xfs/990
> > new file mode 100755
> > index 00000000..3c79186e
> > --- /dev/null
> > +++ b/tests/xfs/990
> > @@ -0,0 +1,70 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021 Red Hat, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test 990
> > +#
> > +# XFS shrinkfs basic functionality test
> > +#
> > +# This test attempts to shrink with a small size (512K), half AG size and
> > +# an out-of-bound size (agsize + 1) to observe if it works as expected.
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +test_shrink()
> > +{
> > + $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT >> $seqres.full 2>&1
> > + ret=$?
>
> I'm not sure what's the output of xfs_growfs when shrinking filesystem,
> if it's easy to do filter, but it'd be good if we could just dump the
> output and let .out file match & check the test result.
Not quite sure if it's of some use (e.g. also need to update expected output
when output changed), since I think just make sure the shrinked size is as
expected and xfs_repair won't argue anything on the new size, that would be
enough.
>
> > +
> > + _scratch_unmount
> > + _repair_scratch_fs >> $seqres.full
>
> _repair_scratch_fs will fix corruption if there's any, and always return 0 if
> completes without problems. Is _check_scratch_fs() what you want?
I didn't notice this, will update.
>
> > + _scratch_mount >> $seqres.full
> > +
> > + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
> > + . $tmp.growfs
> > + [ $ret -eq 0 -a $1 -eq $dblocks ]
>
> Just dump the expected size after shrink if possible.
er... my idea is that I don't want to continue the test case
if any fails... if dump the expected size to output, I need to
fix blocksize as well, not sure if it's necessary.
>
> > +}
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
>
> Still missing _require_scratch
No, _require_xfs_shrink will include that, just as _require_xfs_scratch_rmapbt.
>
> > +_require_xfs_shrink
> > +
> > +rm -f $seqres.full
> > +echo "Format and mount"
> > +_scratch_mkfs -dsize="$((512 * 1024 * 1024))" -dagcount=3 2>&1 | \
> > + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
>
> Better to explain why mkfs with agcount=3
just because agcount = 1 is forbidden on purpose, and I need to make sure
shrinking to 2 AG is not possible yet. So agcount = 3 is the minimum number.
It's just a functionality test, so agcount = 3 also makes the result
determinatively.
>
> > +. $tmp.mkfs
> > +t_dblocks=$dblocks
> > +_scratch_mount >> $seqres.full
> > +
> > +echo "Shrink fs (small size)"
> > +test_shrink $((t_dblocks-512*1024/dbsize)) || \
> > + _fail "Shrink fs (small size) failure"
>
> If it's possible to turn test_shrink to .out file matching way, _fail is
> not needed and below.
The same as above. I could try to fix blocksize if the opinion is strong.
>
> > +
> > +echo "Shrink fs (half AG)"
> > +test_shrink $((t_dblocks-agsize/2)) || \
> > + _fail "Shrink fs (half AG) failure"
> > +
> > +echo "Shrink fs (out-of-bound)"
> > +test_shrink $((t_dblocks-agsize-1)) && \
> > + _fail "Shrink fs (out-of-bound) failure"
> > +[ $dblocks -ne $((t_dblocks-agsize/2)) ] && \
> > + _fail "dblocks changed after shrinking failure"
> > +
> > +$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full
> > +echo "*** done"
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/xfs/990.out b/tests/xfs/990.out
> > new file mode 100644
> > index 00000000..812f89ef
> > --- /dev/null
> > +++ b/tests/xfs/990.out
> > @@ -0,0 +1,12 @@
> > +QA output created by 990
> > +Format and mount
> > +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> > +data = bsize=XXX blocks=XXX, imaxpct=PCT
> > + = sunit=XXX swidth=XXX, unwritten=X
> > +naming =VERN bsize=XXX
> > +log =LDEV bsize=XXX blocks=XXX
> > +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> > +Shrink fs (small size)
> > +Shrink fs (half AG)
> > +Shrink fs (out-of-bound)
> > +*** done
> > diff --git a/tests/xfs/group b/tests/xfs/group
> > index e861cec9..a7981b67 100644
> > --- a/tests/xfs/group
> > +++ b/tests/xfs/group
> > @@ -525,3 +525,4 @@
> > 525 auto quick mkfs
> > 526 auto quick mkfs
> > 527 auto quick quota
> > +990 auto quick growfs
>
> Maybe it's time to add a new 'shrinkfs' group?
not sure if it needs, since shrinkfs reuses growfs ioctl.
Thanks,
Gao Xiang
>
> Thanks,
> Eryu
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 3/3] xfs: stress test for shrinking free space in the last AG
2021-03-28 16:46 ` Eryu Guan
@ 2021-03-28 20:09 ` Gao Xiang
0 siblings, 0 replies; 12+ messages in thread
From: Gao Xiang @ 2021-03-28 20:09 UTC (permalink / raw)
To: Eryu Guan; +Cc: linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 29, 2021 at 12:46:29AM +0800, Eryu Guan wrote:
> On Mon, Mar 15, 2021 at 07:19:26PM +0800, Gao Xiang wrote:
> > This adds a stress testcase to shrink free space as much as
> > possible in the last AG with background fsstress workload.
> >
> > The expectation is that no crash happens with expected output.
> >
> > Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> > ---
> > tests/xfs/991 | 122 ++++++++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/991.out | 8 +++
> > tests/xfs/group | 1 +
> > 3 files changed, 131 insertions(+)
> > create mode 100755 tests/xfs/991
> > create mode 100644 tests/xfs/991.out
> >
> > diff --git a/tests/xfs/991 b/tests/xfs/991
> > new file mode 100755
> > index 00000000..7e7d318e
> > --- /dev/null
> > +++ b/tests/xfs/991
> > @@ -0,0 +1,122 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2020-2021 Red Hat, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test 991
> > +#
> > +# XFS online shrinkfs stress test
> > +#
> > +# This test attempts to shrink unused space as much as possible with
> > +# background fsstress workload. It will decrease the shrink size if
> > +# larger size fails. And totally repeat 2 * TIME_FACTOR times.
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +create_scratch()
> > +{
> > + _scratch_mkfs_xfs $@ | tee -a $seqres.full | \
> > + _filter_mkfs 2>$tmp.mkfs >/dev/null
> > + . $tmp.mkfs
> > +
> > + if ! _try_scratch_mount 2>/dev/null; then
> > + echo "failed to mount $SCRATCH_DEV"
> > + exit 1
> > + fi
>
> _scratch_mount will do the check and _fail the test on mount failure.
It was just copied & pasted from somewhere else rather than wrote from
scratch.
>
> > +
> > + # fix the reserve block pool to a known size so that the enospc
> > + # calculations work out correctly.
> > + _scratch_resvblks 1024 > /dev/null 2>&1
> > +}
> > +
> > +fill_scratch()
> > +{
> > + $XFS_IO_PROG -f -c "falloc -k 0 $1" $SCRATCH_MNT/resvfile
> > +}
> > +
> > +stress_scratch()
> > +{
> > + procs=3
> > + nops=$((1000 * LOAD_FACTOR))
>
> Declare procs and nops as local.
Will update, thanks for the suggestion.
>
> > + # -w ensures that the only ops are ones which cause write I/O
> > + FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -w -p $procs \
> > + -n $nops $FSSTRESS_AVOID`
> > + $FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full 2>&1 &
>
> I think it's more explicit to run run stress_scratch in background,
> instead run fsstress in background implicit.
Will update as well.
>
> > +}
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
>
> _require_scratch
No, _require_xfs_shrink also checks this.
>
> > +_require_xfs_shrink
> > +_require_xfs_io_command "falloc"
> > +
> > +rm -f $seqres.full
> > +_scratch_mkfs_xfs | tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
> > +. $tmp.mkfs # extract blocksize and data size for scratch device
> > +
> > +decsize=`expr 42 \* 1048576` # shrink in chunks of this size at most
> > +endsize=`expr 125 \* 1048576` # stop after shrinking this big
> > +[ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
> > +
> > +nags=2
> > +totalcount=$((2 * TIME_FACTOR))
> > +
> > +while [ $totalcount -gt 0 ]; do
> > + size=`expr 1010 \* 1048576` # 1010 megabytes initially
> > + logblks=$(_scratch_find_xfs_min_logblocks -dsize=${size} -dagcount=${nags})
> > +
> > + create_scratch -lsize=${logblks}b -dsize=${size} -dagcount=${nags}
> > +
> > + for i in `seq 125 -1 90`; do
> > + fillsize=`expr $i \* 1048576`
> > + out="$(fill_scratch $fillsize 2>&1)"
> > + echo "$out" | grep -q 'No space left on device' && continue
> > + test -n "${out}" && echo "$out"
> > + break
> > + done
> > +
> > + while [ $size -gt $endsize ]; do
> > + stress_scratch
>
> So just call
>
> stress_scratch &
>
> here? So it's clear that we put stress_scratch in background, and the
> 'wait' below is waiting for it.
Will update.
Thanks,
Gao Xiang
>
> Thanks,
> Eryu
>
> > + sleep 1
> > +
> > + decb=`expr $decsize / $dbsize` # in data blocks
> > + while [ $decb -gt 0 ]; do
> > + sizeb=`expr $size / $dbsize - $decb`
> > +
> > + $XFS_GROWFS_PROG -D ${sizeb} $SCRATCH_MNT \
> > + >> $seqres.full 2>&1 && break
> > +
> > + [ $decb -gt 100 ] && decb=`expr $decb + $RANDOM % 10`
> > + decb=`expr $decb / 2`
> > + done
> > +
> > + wait
> > + [ $decb -eq 0 ] && break
> > +
> > + # get latest dblocks
> > + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
> > + . $tmp.growfs
> > +
> > + size=`expr $dblocks \* $dbsize`
> > + _scratch_unmount
> > + _repair_scratch_fs >> $seqres.full
> > + _scratch_mount
> > + done
> > +
> > + _scratch_unmount
> > + _repair_scratch_fs >> $seqres.full
> > + totalcount=`expr $totalcount - 1`
> > +done
> > +
> > +echo "*** done"
> > +status=0
> > +exit
> > diff --git a/tests/xfs/991.out b/tests/xfs/991.out
> > new file mode 100644
> > index 00000000..e8209672
> > --- /dev/null
> > +++ b/tests/xfs/991.out
> > @@ -0,0 +1,8 @@
> > +QA output created by 991
> > +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> > +data = bsize=XXX blocks=XXX, imaxpct=PCT
> > + = sunit=XXX swidth=XXX, unwritten=X
> > +naming =VERN bsize=XXX
> > +log =LDEV bsize=XXX blocks=XXX
> > +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> > +*** done
> > diff --git a/tests/xfs/group b/tests/xfs/group
> > index a7981b67..cf190b59 100644
> > --- a/tests/xfs/group
> > +++ b/tests/xfs/group
> > @@ -526,3 +526,4 @@
> > 526 auto quick mkfs
> > 527 auto quick quota
> > 990 auto quick growfs
> > +991 auto growfs ioctl prealloc stress
> > --
> > 2.27.0
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG
2021-03-28 20:06 ` Gao Xiang
@ 2021-03-29 2:14 ` Eryu Guan
2021-03-29 2:43 ` Gao Xiang
0 siblings, 1 reply; 12+ messages in thread
From: Eryu Guan @ 2021-03-29 2:14 UTC (permalink / raw)
To: Gao Xiang; +Cc: Eryu Guan, linux-xfs, fstests, Darrick J. Wong, Zorro Lang
On Mon, Mar 29, 2021 at 04:06:03AM +0800, Gao Xiang wrote:
> On Mon, Mar 29, 2021 at 12:32:39AM +0800, Eryu Guan wrote:
> > On Mon, Mar 15, 2021 at 07:19:25PM +0800, Gao Xiang wrote:
> > > Add basic test to make sure the functionality works as expected.
> > >
> > > Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> > > ---
> > > tests/xfs/990 | 70 +++++++++++++++++++++++++++++++++++++++++++++++
> > > tests/xfs/990.out | 12 ++++++++
> > > tests/xfs/group | 1 +
> > > 3 files changed, 83 insertions(+)
> > > create mode 100755 tests/xfs/990
> > > create mode 100644 tests/xfs/990.out
> > >
> > > diff --git a/tests/xfs/990 b/tests/xfs/990
> > > new file mode 100755
> > > index 00000000..3c79186e
> > > --- /dev/null
> > > +++ b/tests/xfs/990
> > > @@ -0,0 +1,70 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2021 Red Hat, Inc. All Rights Reserved.
> > > +#
> > > +# FS QA Test 990
> > > +#
> > > +# XFS shrinkfs basic functionality test
> > > +#
> > > +# This test attempts to shrink with a small size (512K), half AG size and
> > > +# an out-of-bound size (agsize + 1) to observe if it works as expected.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +
> > > +test_shrink()
> > > +{
> > > + $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT >> $seqres.full 2>&1
> > > + ret=$?
> >
> > I'm not sure what's the output of xfs_growfs when shrinking filesystem,
> > if it's easy to do filter, but it'd be good if we could just dump the
> > output and let .out file match & check the test result.
>
> Not quite sure if it's of some use (e.g. also need to update expected output
> when output changed), since I think just make sure the shrinked size is as
> expected and xfs_repair won't argue anything on the new size, that would be
> enough.
>
> >
> > > +
> > > + _scratch_unmount
> > > + _repair_scratch_fs >> $seqres.full
> >
> > _repair_scratch_fs will fix corruption if there's any, and always return 0 if
> > completes without problems. Is _check_scratch_fs() what you want?
>
> I didn't notice this, will update.
>
> >
> > > + _scratch_mount >> $seqres.full
> > > +
> > > + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
> > > + . $tmp.growfs
> > > + [ $ret -eq 0 -a $1 -eq $dblocks ]
> >
> > Just dump the expected size after shrink if possible.
>
> er... my idea is that I don't want to continue the test case
> if any fails... if dump the expected size to output, I need to
> fix blocksize as well, not sure if it's necessary.
>
> >
> > > +}
> > > +
> > > +# real QA test starts here
> > > +_supported_fs xfs
> >
> > Still missing _require_scratch
>
> No, _require_xfs_shrink will include that, just as _require_xfs_scratch_rmapbt.
Then include the "scratch" part in the helper name, so it's clear that
it requires the SCRATCH_DEV, i.e. _require_xfs_scratch_shrink
>
> >
> > > +_require_xfs_shrink
> > > +
> > > +rm -f $seqres.full
> > > +echo "Format and mount"
> > > +_scratch_mkfs -dsize="$((512 * 1024 * 1024))" -dagcount=3 2>&1 | \
> > > + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
> >
> > Better to explain why mkfs with agcount=3
>
> just because agcount = 1 is forbidden on purpose, and I need to make sure
> shrinking to 2 AG is not possible yet. So agcount = 3 is the minimum number.
>
> It's just a functionality test, so agcount = 3 also makes the result
> determinatively.
Yeah, I'd guess the reason, but it'd be great to make it clear in
comments.
>
> >
> > > +. $tmp.mkfs
> > > +t_dblocks=$dblocks
> > > +_scratch_mount >> $seqres.full
> > > +
> > > +echo "Shrink fs (small size)"
> > > +test_shrink $((t_dblocks-512*1024/dbsize)) || \
> > > + _fail "Shrink fs (small size) failure"
> >
> > If it's possible to turn test_shrink to .out file matching way, _fail is
> > not needed and below.
>
> The same as above. I could try to fix blocksize if the opinion is strong.
>
> >
> > > +
> > > +echo "Shrink fs (half AG)"
> > > +test_shrink $((t_dblocks-agsize/2)) || \
> > > + _fail "Shrink fs (half AG) failure"
> > > +
> > > +echo "Shrink fs (out-of-bound)"
> > > +test_shrink $((t_dblocks-agsize-1)) && \
> > > + _fail "Shrink fs (out-of-bound) failure"
> > > +[ $dblocks -ne $((t_dblocks-agsize/2)) ] && \
> > > + _fail "dblocks changed after shrinking failure"
> > > +
> > > +$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full
> > > +echo "*** done"
> > > +
> > > +# success, all done
> > > +status=0
> > > +exit
> > > diff --git a/tests/xfs/990.out b/tests/xfs/990.out
> > > new file mode 100644
> > > index 00000000..812f89ef
> > > --- /dev/null
> > > +++ b/tests/xfs/990.out
> > > @@ -0,0 +1,12 @@
> > > +QA output created by 990
> > > +Format and mount
> > > +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> > > +data = bsize=XXX blocks=XXX, imaxpct=PCT
> > > + = sunit=XXX swidth=XXX, unwritten=X
> > > +naming =VERN bsize=XXX
> > > +log =LDEV bsize=XXX blocks=XXX
> > > +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> > > +Shrink fs (small size)
> > > +Shrink fs (half AG)
> > > +Shrink fs (out-of-bound)
> > > +*** done
> > > diff --git a/tests/xfs/group b/tests/xfs/group
> > > index e861cec9..a7981b67 100644
> > > --- a/tests/xfs/group
> > > +++ b/tests/xfs/group
> > > @@ -525,3 +525,4 @@
> > > 525 auto quick mkfs
> > > 526 auto quick mkfs
> > > 527 auto quick quota
> > > +990 auto quick growfs
> >
> > Maybe it's time to add a new 'shrinkfs' group?
>
> not sure if it needs, since shrinkfs reuses growfs ioctl.
Ok, I have no strong preference on this, just think it's a bit easier to
run only shrink tests by ./check -g shrinkfs.
Thanks,
Eryu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG
2021-03-29 2:14 ` Eryu Guan
@ 2021-03-29 2:43 ` Gao Xiang
0 siblings, 0 replies; 12+ messages in thread
From: Gao Xiang @ 2021-03-29 2:43 UTC (permalink / raw)
To: Eryu Guan; +Cc: Eryu Guan, linux-xfs, fstests, Darrick J. Wong, Zorro Lang
Hi Eryu,
On Mon, Mar 29, 2021 at 10:14:26AM +0800, Eryu Guan wrote:
> On Mon, Mar 29, 2021 at 04:06:03AM +0800, Gao Xiang wrote:
> > On Mon, Mar 29, 2021 at 12:32:39AM +0800, Eryu Guan wrote:
> > > On Mon, Mar 15, 2021 at 07:19:25PM +0800, Gao Xiang wrote:
> > > > Add basic test to make sure the functionality works as expected.
> > > >
> > > > Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> > > > ---
> > > > tests/xfs/990 | 70 +++++++++++++++++++++++++++++++++++++++++++++++
> > > > tests/xfs/990.out | 12 ++++++++
> > > > tests/xfs/group | 1 +
> > > > 3 files changed, 83 insertions(+)
> > > > create mode 100755 tests/xfs/990
> > > > create mode 100644 tests/xfs/990.out
> > > >
> > > > diff --git a/tests/xfs/990 b/tests/xfs/990
> > > > new file mode 100755
> > > > index 00000000..3c79186e
> > > > --- /dev/null
> > > > +++ b/tests/xfs/990
> > > > @@ -0,0 +1,70 @@
> > > > +#! /bin/bash
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +# Copyright (c) 2021 Red Hat, Inc. All Rights Reserved.
> > > > +#
> > > > +# FS QA Test 990
> > > > +#
> > > > +# XFS shrinkfs basic functionality test
> > > > +#
> > > > +# This test attempts to shrink with a small size (512K), half AG size and
> > > > +# an out-of-bound size (agsize + 1) to observe if it works as expected.
> > > > +#
> > > > +seq=`basename $0`
> > > > +seqres=$RESULT_DIR/$seq
> > > > +echo "QA output created by $seq"
> > > > +
> > > > +here=`pwd`
> > > > +tmp=/tmp/$$
> > > > +status=1 # failure is the default!
> > > > +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> > > > +
> > > > +# get standard environment, filters and checks
> > > > +. ./common/rc
> > > > +. ./common/filter
> > > > +
> > > > +test_shrink()
> > > > +{
> > > > + $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT >> $seqres.full 2>&1
> > > > + ret=$?
> > >
> > > I'm not sure what's the output of xfs_growfs when shrinking filesystem,
> > > if it's easy to do filter, but it'd be good if we could just dump the
> > > output and let .out file match & check the test result.
> >
> > Not quite sure if it's of some use (e.g. also need to update expected output
> > when output changed), since I think just make sure the shrinked size is as
> > expected and xfs_repair won't argue anything on the new size, that would be
> > enough.
> >
> > >
> > > > +
> > > > + _scratch_unmount
> > > > + _repair_scratch_fs >> $seqres.full
> > >
> > > _repair_scratch_fs will fix corruption if there's any, and always return 0 if
> > > completes without problems. Is _check_scratch_fs() what you want?
> >
> > I didn't notice this, will update.
> >
> > >
> > > > + _scratch_mount >> $seqres.full
> > > > +
> > > > + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null
> > > > + . $tmp.growfs
> > > > + [ $ret -eq 0 -a $1 -eq $dblocks ]
> > >
> > > Just dump the expected size after shrink if possible.
> >
> > er... my idea is that I don't want to continue the test case
> > if any fails... if dump the expected size to output, I need to
> > fix blocksize as well, not sure if it's necessary.
> >
> > >
> > > > +}
> > > > +
> > > > +# real QA test starts here
> > > > +_supported_fs xfs
> > >
> > > Still missing _require_scratch
> >
> > No, _require_xfs_shrink will include that, just as _require_xfs_scratch_rmapbt.
>
> Then include the "scratch" part in the helper name, so it's clear that
> it requires the SCRATCH_DEV, i.e. _require_xfs_scratch_shrink
ok, will rename this later. I'm working on the other stuff now.
>
> >
> > >
> > > > +_require_xfs_shrink
> > > > +
> > > > +rm -f $seqres.full
> > > > +echo "Format and mount"
> > > > +_scratch_mkfs -dsize="$((512 * 1024 * 1024))" -dagcount=3 2>&1 | \
> > > > + tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
> > >
> > > Better to explain why mkfs with agcount=3
> >
> > just because agcount = 1 is forbidden on purpose, and I need to make sure
> > shrinking to 2 AG is not possible yet. So agcount = 3 is the minimum number.
> >
> > It's just a functionality test, so agcount = 3 also makes the result
> > determinatively.
>
> Yeah, I'd guess the reason, but it'd be great to make it clear in
> comments.
ok, will try to add some description.
>
> >
> > >
> > > > +. $tmp.mkfs
> > > > +t_dblocks=$dblocks
> > > > +_scratch_mount >> $seqres.full
> > > > +
> > > > +echo "Shrink fs (small size)"
> > > > +test_shrink $((t_dblocks-512*1024/dbsize)) || \
> > > > + _fail "Shrink fs (small size) failure"
> > >
> > > If it's possible to turn test_shrink to .out file matching way, _fail is
> > > not needed and below.
> >
> > The same as above. I could try to fix blocksize if the opinion is strong.
> >
> > >
> > > > +
> > > > +echo "Shrink fs (half AG)"
> > > > +test_shrink $((t_dblocks-agsize/2)) || \
> > > > + _fail "Shrink fs (half AG) failure"
> > > > +
> > > > +echo "Shrink fs (out-of-bound)"
> > > > +test_shrink $((t_dblocks-agsize-1)) && \
> > > > + _fail "Shrink fs (out-of-bound) failure"
> > > > +[ $dblocks -ne $((t_dblocks-agsize/2)) ] && \
> > > > + _fail "dblocks changed after shrinking failure"
> > > > +
> > > > +$XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full
> > > > +echo "*** done"
> > > > +
> > > > +# success, all done
> > > > +status=0
> > > > +exit
> > > > diff --git a/tests/xfs/990.out b/tests/xfs/990.out
> > > > new file mode 100644
> > > > index 00000000..812f89ef
> > > > --- /dev/null
> > > > +++ b/tests/xfs/990.out
> > > > @@ -0,0 +1,12 @@
> > > > +QA output created by 990
> > > > +Format and mount
> > > > +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> > > > +data = bsize=XXX blocks=XXX, imaxpct=PCT
> > > > + = sunit=XXX swidth=XXX, unwritten=X
> > > > +naming =VERN bsize=XXX
> > > > +log =LDEV bsize=XXX blocks=XXX
> > > > +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> > > > +Shrink fs (small size)
> > > > +Shrink fs (half AG)
> > > > +Shrink fs (out-of-bound)
> > > > +*** done
> > > > diff --git a/tests/xfs/group b/tests/xfs/group
> > > > index e861cec9..a7981b67 100644
> > > > --- a/tests/xfs/group
> > > > +++ b/tests/xfs/group
> > > > @@ -525,3 +525,4 @@
> > > > 525 auto quick mkfs
> > > > 526 auto quick mkfs
> > > > 527 auto quick quota
> > > > +990 auto quick growfs
> > >
> > > Maybe it's time to add a new 'shrinkfs' group?
> >
> > not sure if it needs, since shrinkfs reuses growfs ioctl.
>
> Ok, I have no strong preference on this, just think it's a bit easier to
> run only shrink tests by ./check -g shrinkfs.
Currently only 2 test cases are available. ok, if that is needed, I could
add a new shrinkfs group.
(Also noted that xfsprogs patch haven't been merged yet. So I think the
testcase needs to be formally merged after xfsprogs side is settled.)
Thanks,
Gao Xiang
>
> Thanks,
> Eryu
>
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-03-29 2:43 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-15 11:19 [RFC PATCH v3 0/3] xfs: testcases for shrinking free space in the last AG Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 1/3] common/xfs: add a _require_xfs_shrink helper Gao Xiang
2021-03-28 16:18 ` Eryu Guan
2021-03-28 19:51 ` Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 2/3] xfs: basic functionality test for shrinking free space in the last AG Gao Xiang
2021-03-28 16:32 ` Eryu Guan
2021-03-28 20:06 ` Gao Xiang
2021-03-29 2:14 ` Eryu Guan
2021-03-29 2:43 ` Gao Xiang
2021-03-15 11:19 ` [RFC PATCH v3 3/3] xfs: stress " Gao Xiang
2021-03-28 16:46 ` Eryu Guan
2021-03-28 20:09 ` Gao Xiang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.