[PATCH V4 00/11] xfs: Tests to verify inode fork extent count overflow detection

[PATCH V4 00/11] xfs: Tests to verify inode fork extent count overflow detection

[Chandan Babu R]

The patchset at
https://lore.kernel.org/linux-xfs/20210110160720.3922965-1-chandanrlinux@gmail.com/T/
added support for XFS to detect inode extent count overflow when
performing various filesystem operations. The patchset also added
new error injection tags for,
1. Reducing maximum extent count to 10.
2. Allocating only single block sized extents.

The corresponding code for xfsprogs can be obtained from
https://lore.kernel.org/linux-xfs/20201104114900.172147-1-chandanrlinux@gmail.com/.

The patches posted along with this cover letter add tests to verify
the correctness of in-kernel inode extent count overflow detection
mechanism.

These patches can also be obtained from
https://github.com/chandanr/xfstests.git at branch
extent-overflow-tests.

Changelog:
V3 -> V4:
  1. Rebase on fstests' latest master branch.

V2 -> V3:
  1. Kernel's max pseudo extent count is once again set to 10. Hence
     change the tests back to using 10 as the maximum extent count.
  2. The following tests associated with directories have been removed
     since directory remove and rename operations are not expected to
     fail (with -EFBIG error code) in low inode extent count
     scenarios:
     - Populate source directory and mv one entry to destination
       directory.
     - Populate a directory and remove one entry.

V1 -> V2:
  1. Obtain extent count for inodes accessible through filesystem path
     names via "xfs_io -c 'stat' ..." command line. This gets rid of
     unmount/mount cycles from most of the tests.
  2. Use _fill_fs() to consume free space of a filesystem.
  3. Use _scratch_inject_error() helper to enable/disable error tags.
  4. Use sizeof(struct xfs_dqblk) to calculate number of quotas inside
     one filesystem block.
  5. Write once to every block of the quota inode instead of
     sequentially filling up each block.
  6. Use _get_file_block_size() for tests involving regular files.
  7. Modify tests to suit the new pseudo max extent count of 35.
  8. Replace xfs_io with $XFS_IO_PROG.
  9. Remove code that extended a realtime file since this takes the
     same path as direct I/O to a regular file.
  10. For xattr, do not execute test script when block size is < 4k.
  11. Add code to test if removing an xattr from a full attribute fork
      succeeds.
  12. Add code to test if removing a file whose attribute fork is full
      succeeds.
  13. Add code to test if removing a entry from a full directory
      succeeds.
  14. Add code to test if removing a full directory succeeds.
  15. Writing to unwritten extents: Integrate buffered and direct I/O
      tests into a for loop.
  16. Writing to a shared extent test: Add test to check funshare
      operation.
  17. Use _scale_fsstress_args() to scale values for "-p" and "-n"
      arguments for fsstress.

Chandan Babu R (11):
  common/xfs: Add a helper to get an inode fork's extent count
  xfs: Check for extent overflow when trivally adding a new extent
  xfs: Check for extent overflow when growing realtime bitmap/summary
    inodes
  xfs: Check for extent overflow when punching a hole
  xfs: Check for extent overflow when adding/removing xattrs
  xfs: Check for extent overflow when adding/removing dir entries
  xfs: Check for extent overflow when writing to unwritten extent
  xfs: Check for extent overflow when moving extent from cow to data
    fork
  xfs: Check for extent overflow when remapping an extent
  xfs: Check for extent overflow when swapping extents
  xfs: Stress test with bmap_alloc_minlen_extent error tag enabled

 common/xfs        |  22 ++++++
 tests/xfs/522     | 173 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/522.out |  20 +++++
 tests/xfs/523     | 119 +++++++++++++++++++++++++++++
 tests/xfs/523.out |  11 +++
 tests/xfs/524     |  84 +++++++++++++++++++++
 tests/xfs/524.out |  19 +++++
 tests/xfs/525     | 141 +++++++++++++++++++++++++++++++++++
 tests/xfs/525.out |  18 +++++
 tests/xfs/526     | 186 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/526.out |  17 +++++
 tests/xfs/527     |  89 ++++++++++++++++++++++
 tests/xfs/527.out |  11 +++
 tests/xfs/528     | 110 +++++++++++++++++++++++++++
 tests/xfs/528.out |  12 +++
 tests/xfs/529     |  82 ++++++++++++++++++++
 tests/xfs/529.out |   8 ++
 tests/xfs/530     | 109 +++++++++++++++++++++++++++
 tests/xfs/530.out |  13 ++++
 tests/xfs/531     |  84 +++++++++++++++++++++
 tests/xfs/531.out |   7 ++
 tests/xfs/group   |  10 +++
 22 files changed, 1345 insertions(+)
 create mode 100755 tests/xfs/522
 create mode 100644 tests/xfs/522.out
 create mode 100755 tests/xfs/523
 create mode 100644 tests/xfs/523.out
 create mode 100755 tests/xfs/524
 create mode 100644 tests/xfs/524.out
 create mode 100755 tests/xfs/525
 create mode 100644 tests/xfs/525.out
 create mode 100755 tests/xfs/526
 create mode 100644 tests/xfs/526.out
 create mode 100755 tests/xfs/527
 create mode 100644 tests/xfs/527.out
 create mode 100755 tests/xfs/528
 create mode 100644 tests/xfs/528.out
 create mode 100755 tests/xfs/529
 create mode 100644 tests/xfs/529.out
 create mode 100755 tests/xfs/530
 create mode 100644 tests/xfs/530.out
 create mode 100755 tests/xfs/531
 create mode 100644 tests/xfs/531.out

-- 
2.29.2

[PATCH V4 01/11] common/xfs: Add a helper to get an inode fork's extent count

[Chandan Babu R]

This commit adds the helper _scratch_get_iext_count() which returns an
inode fork's extent count.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 common/xfs | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/common/xfs b/common/xfs
index 3f5c14ba..641d6195 100644
--- a/common/xfs
+++ b/common/xfs
@@ -903,6 +903,28 @@ _scratch_get_bmx_prefix() {
 	return 1
 }
 
+_scratch_get_iext_count()
+{
+	ino=$1
+	whichfork=$2
+
+	case $whichfork in
+		"attr")
+			field=core.naextents
+			;;
+		"data")
+			field=core.nextents
+			;;
+		*)
+			return 1
+	esac
+
+	nextents=$(_scratch_xfs_db  -c "inode $ino" -c "print $field")
+	nextents=${nextents##${field} = }
+
+	echo $nextents
+}
+
 #
 # Ensures that we don't pass any mount options incompatible with XFS v4
 #
-- 
2.29.2

[PATCH V4 02/11] xfs: Check for extent overflow when trivally adding a new extent

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when adding a single extent while there's no possibility of
splitting an existing mapping.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/522     | 173 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/522.out |  20 ++++++
 tests/xfs/group   |   1 +
 3 files changed, 194 insertions(+)
 create mode 100755 tests/xfs/522
 create mode 100644 tests/xfs/522.out

diff --git a/tests/xfs/522 b/tests/xfs/522
new file mode 100755
index 00000000..33f0591e
--- /dev/null
+++ b/tests/xfs/522
@@ -0,0 +1,173 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 522
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# adding a single extent while there's no possibility of splitting an existing
+# mapping.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/quota
+. ./common/inject
+. ./common/populate
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_xfs_quota
+_require_xfs_debug
+_require_test_program "punch-alternating"
+_require_xfs_io_command "falloc"
+_require_xfs_io_error_injection "reduce_max_iextents"
+_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
+
+echo "Format and mount fs"
+_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full
+_scratch_mount -o uquota >> $seqres.full
+
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+
+echo "* Delalloc to written extent conversion"
+
+testfile=$SCRATCH_MNT/testfile
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+nr_blks=$((15 * 2))
+
+echo "Create fragmented file"
+for i in $(seq 0 2 $((nr_blks - 1))); do
+	$XFS_IO_PROG -f -s -c "pwrite $((i * bsize)) $bsize" $testfile \
+	       >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+echo "Verify \$testfile's extent count"
+
+nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm $testfile
+
+echo "* Fallocate unwritten extents"
+
+echo "Fallocate fragmented file"
+for i in $(seq 0 2 $((nr_blks - 1))); do
+	$XFS_IO_PROG -f -c "falloc $((i * bsize)) $bsize" $testfile \
+	       >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+echo "Verify \$testfile's extent count"
+
+nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm $testfile
+
+echo "* Directio write"
+
+echo "Create fragmented file via directio writes"
+for i in $(seq 0 2 $((nr_blks - 1))); do
+	$XFS_IO_PROG -d -s -f -c "pwrite $((i * bsize)) $bsize" $testfile \
+	       >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+echo "Verify \$testfile's extent count"
+
+nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm $testfile
+
+echo "* Extend quota inodes"
+
+echo "Disable reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 0
+
+echo "Consume free space"
+fillerdir=$SCRATCH_MNT/fillerdir
+nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
+nr_free_blks=$((nr_free_blks * 90 / 100))
+
+_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
+
+echo "Create fragmented filesystem"
+for dentry in $(ls -1 $fillerdir/); do
+	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
+done
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Inject bmap_alloc_minlen_extent error tag"
+_scratch_inject_error bmap_alloc_minlen_extent 1
+
+nr_blks=20
+
+# This is a rough calculation; It doesn't take block headers into
+# consideration.
+# gdb -batch vmlinux -ex 'print sizeof(struct xfs_dqblk)'
+# $1 = 136
+nr_quotas_per_block=$((bsize / 136))
+nr_quotas=$((nr_quotas_per_block * nr_blks))
+
+echo "Extend uquota file"
+for i in $(seq 0 $nr_quotas_per_block $nr_quotas); do
+	chown $i $testfile >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+_scratch_unmount >> $seqres.full
+
+echo "Verify uquota inode's extent count"
+uquotino=$(_scratch_xfs_db -c sb -c "print uquotino")
+uquotino=${uquotino##uquotino = }
+
+nextents=$(_scratch_get_iext_count $uquotino data || \
+		   _fail "Unable to obtain inode fork's extent count")
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/522.out b/tests/xfs/522.out
new file mode 100644
index 00000000..debeb004
--- /dev/null
+++ b/tests/xfs/522.out
@@ -0,0 +1,20 @@
+QA output created by 522
+Format and mount fs
+* Delalloc to written extent conversion
+Inject reduce_max_iextents error tag
+Create fragmented file
+Verify $testfile's extent count
+* Fallocate unwritten extents
+Fallocate fragmented file
+Verify $testfile's extent count
+* Directio write
+Create fragmented file via directio writes
+Verify $testfile's extent count
+* Extend quota inodes
+Disable reduce_max_iextents error tag
+Consume free space
+Create fragmented filesystem
+Inject reduce_max_iextents error tag
+Inject bmap_alloc_minlen_extent error tag
+Extend uquota file
+Verify uquota inode's extent count
diff --git a/tests/xfs/group b/tests/xfs/group
index b89c0a4e..1831f0b5 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -519,3 +519,4 @@
 519 auto quick reflink
 520 auto quick reflink
 521 auto quick realtime growfs
+522 auto quick quota
-- 
2.29.2

[PATCH V4 03/11] xfs: Check for extent overflow when growing realtime bitmap/summary inodes

[Chandan Babu R]

Verify that XFS does not cause realtime bitmap/summary inode fork's
extent count to overflow when growing the realtime volume associated
with a filesystem.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/523     | 119 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/523.out |  11 +++++
 tests/xfs/group   |   1 +
 3 files changed, 131 insertions(+)
 create mode 100755 tests/xfs/523
 create mode 100644 tests/xfs/523.out

diff --git a/tests/xfs/523 b/tests/xfs/523
new file mode 100755
index 00000000..145f0ff6
--- /dev/null
+++ b/tests/xfs/523
@@ -0,0 +1,119 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 523
+#
+# Verify that XFS does not cause bitmap/summary inode fork's extent count to
+# overflow when growing an the realtime volume of the filesystem.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	_scratch_unmount >> $seqres.full 2>&1
+	test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1
+	rm -f $tmp.* $TEST_DIR/$seq.rtvol
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/inject
+. ./common/populate
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_test
+_require_xfs_debug
+_require_test_program "punch-alternating"
+_require_xfs_io_error_injection "reduce_max_iextents"
+_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
+_require_scratch_nocheck
+
+echo "* Test extending rt inodes"
+
+_scratch_mkfs | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
+. $tmp.mkfs
+
+echo "Create fake rt volume"
+nr_bitmap_blks=25
+nr_bits=$((nr_bitmap_blks * dbsize * 8))
+rtextsz=$dbsize
+rtdevsz=$((nr_bits * rtextsz))
+truncate -s $rtdevsz $TEST_DIR/$seq.rtvol
+rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol)
+
+echo "Format and mount rt volume"
+
+export USE_EXTERNAL=yes
+export SCRATCH_RTDEV=$rtdev
+_scratch_mkfs -d size=$((1024 * 1024 * 1024)) \
+	      -r size=2M,extsize=${rtextsz} >> $seqres.full
+_scratch_mount >> $seqres.full
+
+echo "Consume free space"
+fillerdir=$SCRATCH_MNT/fillerdir
+nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
+nr_free_blks=$((nr_free_blks * 90 / 100))
+
+_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
+
+echo "Create fragmented filesystem"
+for dentry in $(ls -1 $fillerdir/); do
+	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
+done
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Inject bmap_alloc_minlen_extent error tag"
+_scratch_inject_error bmap_alloc_minlen_extent 1
+
+echo "Grow realtime volume"
+$XFS_GROWFS_PROG -r $SCRATCH_MNT >> $seqres.full 2>&1
+if [[ $? == 0 ]]; then
+	echo "Growfs succeeded; should have failed."
+	exit 1
+fi
+
+_scratch_unmount >> $seqres.full
+
+echo "Verify rbmino's and rsumino's extent count"
+for rtino in rbmino rsumino; do
+	ino=$(_scratch_xfs_db -c sb -c "print $rtino")
+	ino=${ino##${rtino} = }
+	echo "$rtino = $ino" >> $seqres.full
+
+	nextents=$(_scratch_get_iext_count $ino data || \
+			_fail "Unable to obtain inode fork's extent count")
+	if (( $nextents > 10 )); then
+		echo "Extent count overflow check failed: nextents = $nextents"
+		exit 1
+	fi
+done
+
+echo "Check filesystem"
+_check_xfs_filesystem $SCRATCH_DEV none $rtdev
+
+losetup -d $rtdev
+rm -f $TEST_DIR/$seq.rtvol
+
+export USE_EXTERNAL=""
+export SCRATCH_RTDEV=""
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/523.out b/tests/xfs/523.out
new file mode 100644
index 00000000..7df02970
--- /dev/null
+++ b/tests/xfs/523.out
@@ -0,0 +1,11 @@
+QA output created by 523
+* Test extending rt inodes
+Create fake rt volume
+Format and mount rt volume
+Consume free space
+Create fragmented filesystem
+Inject reduce_max_iextents error tag
+Inject bmap_alloc_minlen_extent error tag
+Grow realtime volume
+Verify rbmino's and rsumino's extent count
+Check filesystem
diff --git a/tests/xfs/group b/tests/xfs/group
index 1831f0b5..018c70ef 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -520,3 +520,4 @@
 520 auto quick reflink
 521 auto quick realtime growfs
 522 auto quick quota
+523 auto quick realtime growfs
-- 
2.29.2

[PATCH V4 04/11] xfs: Check for extent overflow when punching a hole

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when punching out an extent.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/524     | 84 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/524.out | 19 +++++++++++
 tests/xfs/group   |  1 +
 3 files changed, 104 insertions(+)
 create mode 100755 tests/xfs/524
 create mode 100644 tests/xfs/524.out

diff --git a/tests/xfs/524 b/tests/xfs/524
new file mode 100755
index 00000000..79fa31d8
--- /dev/null
+++ b/tests/xfs/524
@@ -0,0 +1,84 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 524
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# punching out an extent.
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/inject
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_xfs_debug
+_require_xfs_io_command "fpunch"
+_require_xfs_io_command "finsert"
+_require_xfs_io_command "fcollapse"
+_require_xfs_io_command "fzero"
+_require_xfs_io_error_injection "reduce_max_iextents"
+
+echo "Format and mount fs"
+_scratch_mkfs >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+nr_blks=30
+
+testfile=$SCRATCH_MNT/testfile
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+for op in fpunch finsert fcollapse fzero; do
+	echo "* $op regular file"
+
+	echo "Create \$testfile"
+	$XFS_IO_PROG -f -s \
+		     -c "pwrite -b $((nr_blks * bsize)) 0 $((nr_blks * bsize))" \
+		     $testfile  >> $seqres.full
+
+	echo "$op alternating blocks"
+	for i in $(seq 1 2 $((nr_blks - 1))); do
+		$XFS_IO_PROG -f -c "$op $((i * bsize)) $bsize" $testfile \
+		       >> $seqres.full 2>&1
+		[[ $? != 0 ]] && break
+	done
+
+	echo "Verify \$testfile's extent count"
+
+	nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
+	nextents=${nextents##fsxattr.nextents = }
+	if (( $nextents > 10 )); then
+		echo "Extent count overflow check failed: nextents = $nextents"
+		exit 1
+	fi
+
+	rm $testfile
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/524.out b/tests/xfs/524.out
new file mode 100644
index 00000000..a957f9c7
--- /dev/null
+++ b/tests/xfs/524.out
@@ -0,0 +1,19 @@
+QA output created by 524
+Format and mount fs
+Inject reduce_max_iextents error tag
+* fpunch regular file
+Create $testfile
+fpunch alternating blocks
+Verify $testfile's extent count
+* finsert regular file
+Create $testfile
+finsert alternating blocks
+Verify $testfile's extent count
+* fcollapse regular file
+Create $testfile
+fcollapse alternating blocks
+Verify $testfile's extent count
+* fzero regular file
+Create $testfile
+fzero alternating blocks
+Verify $testfile's extent count
diff --git a/tests/xfs/group b/tests/xfs/group
index 018c70ef..3fa38c36 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -521,3 +521,4 @@
 521 auto quick realtime growfs
 522 auto quick quota
 523 auto quick realtime growfs
+524 auto quick punch zero insert collapse
-- 
2.29.2

[PATCH V4 05/11] xfs: Check for extent overflow when adding/removing xattrs

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when adding/removing xattrs.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/525     | 141 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/525.out |  18 ++++++
 tests/xfs/group   |   1 +
 3 files changed, 160 insertions(+)
 create mode 100755 tests/xfs/525
 create mode 100644 tests/xfs/525.out

diff --git a/tests/xfs/525 b/tests/xfs/525
new file mode 100755
index 00000000..bdca846d
--- /dev/null
+++ b/tests/xfs/525
@@ -0,0 +1,141 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 525
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# Adding/removing xattrs.
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+. ./common/inject
+. ./common/populate
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_attrs
+_require_xfs_debug
+_require_test_program "punch-alternating"
+_require_xfs_io_error_injection "reduce_max_iextents"
+_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
+
+echo "Format and mount fs"
+_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_block_size $SCRATCH_MNT)
+
+attr_len=255
+
+testfile=$SCRATCH_MNT/testfile
+
+echo "Consume free space"
+fillerdir=$SCRATCH_MNT/fillerdir
+nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
+nr_free_blks=$((nr_free_blks * 90 / 100))
+
+_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
+
+echo "Create fragmented filesystem"
+for dentry in $(ls -1 $fillerdir/); do
+	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
+done
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Inject bmap_alloc_minlen_extent error tag"
+_scratch_inject_error bmap_alloc_minlen_extent 1
+
+echo "* Set xattrs"
+
+echo "Create \$testfile"
+touch $testfile
+
+echo "Create xattrs"
+nr_attrs=$((bsize * 20 / attr_len))
+for i in $(seq 1 $nr_attrs); do
+	attr="$(printf "trusted.%0247d" $i)"
+	$SETFATTR_PROG -n "$attr" $testfile >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+echo "Verify \$testfile's naextent count"
+
+naextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep naextents)
+naextents=${naextents##fsxattr.naextents = }
+if (( $naextents > 10 )); then
+	echo "Extent count overflow check failed: naextents = $naextents"
+	exit 1
+fi
+
+echo "Remove \$testfile"
+rm $testfile
+
+echo "* Remove xattrs"
+
+echo "Create \$testfile"
+touch $testfile
+
+echo "Disable reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 0
+
+echo "Create initial xattr extents"
+
+naextents=0
+last=""
+start=1
+nr_attrs=$((bsize / attr_len))
+
+while (( $naextents < 4 )); do
+	end=$((start + nr_attrs - 1))
+
+	for i in $(seq $start $end); do
+		attr="$(printf "trusted.%0247d" $i)"
+		$SETFATTR_PROG -n $attr $testfile
+	done
+
+	start=$((end + 1))
+
+	naextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep naextents)
+	naextents=${naextents##fsxattr.naextents = }
+done
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Remove xattr to trigger -EFBIG"
+attr="$(printf "trusted.%0247d" 1)"
+$SETFATTR_PROG -x "$attr" $testfile >> $seqres.full 2>&1
+if [[ $? == 0 ]]; then
+	echo "Xattr removal succeeded; Should have failed "
+	exit 1
+fi
+
+rm $testfile && echo "Successfully removed \$testfile"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/525.out b/tests/xfs/525.out
new file mode 100644
index 00000000..74b152d9
--- /dev/null
+++ b/tests/xfs/525.out
@@ -0,0 +1,18 @@
+QA output created by 525
+Format and mount fs
+Consume free space
+Create fragmented filesystem
+Inject reduce_max_iextents error tag
+Inject bmap_alloc_minlen_extent error tag
+* Set xattrs
+Create $testfile
+Create xattrs
+Verify $testfile's naextent count
+Remove $testfile
+* Remove xattrs
+Create $testfile
+Disable reduce_max_iextents error tag
+Create initial xattr extents
+Inject reduce_max_iextents error tag
+Remove xattr to trigger -EFBIG
+Successfully removed $testfile
diff --git a/tests/xfs/group b/tests/xfs/group
index 3fa38c36..bd38aff0 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -522,3 +522,4 @@
 522 auto quick quota
 523 auto quick realtime growfs
 524 auto quick punch zero insert collapse
+525 auto quick attr
-- 
2.29.2

[PATCH V4 06/11] xfs: Check for extent overflow when adding/removing dir entries

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when adding/removing directory entries.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/526     | 186 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/526.out |  17 +++++
 tests/xfs/group   |   1 +
 3 files changed, 204 insertions(+)
 create mode 100755 tests/xfs/526
 create mode 100644 tests/xfs/526.out

diff --git a/tests/xfs/526 b/tests/xfs/526
new file mode 100755
index 00000000..5a789d61
--- /dev/null
+++ b/tests/xfs/526
@@ -0,0 +1,186 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 526
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# adding/removing directory entries.
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/inject
+. ./common/populate
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_xfs_debug
+_require_test_program "punch-alternating"
+_require_xfs_io_error_injection "reduce_max_iextents"
+_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
+
+_scratch_mkfs_sized $((1024 * 1024 * 1024)) | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
+. $tmp.mkfs
+
+# Filesystems with directory block size greater than one FSB will not be tested,
+# since "7 (i.e. XFS_DA_NODE_MAXDEPTH + 1 data block + 1 free block) * 2 (fsb
+# count) = 14" is greater than the pseudo max extent count limit of 10.
+# Extending the pseudo max limit won't help either.  Consider the case where 1
+# FSB is 1k in size and 1 dir block is 64k in size (i.e. fsb count = 64). In
+# this case, the pseudo max limit has to be greater than 7 * 64 = 448 extents.
+if (( $dbsize != $dirbsize )); then
+	_notrun "FSB size ($dbsize) and directory block size ($dirbsize) do not match"
+fi
+
+echo "Format and mount fs"
+_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
+_scratch_mount >> $seqres.full
+
+echo "Consume free space"
+fillerdir=$SCRATCH_MNT/fillerdir
+nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
+nr_free_blks=$((nr_free_blks * 90 / 100))
+
+_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
+
+echo "Create fragmented filesystem"
+for dentry in $(ls -1 $fillerdir/); do
+	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
+done
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Inject bmap_alloc_minlen_extent error tag"
+_scratch_inject_error bmap_alloc_minlen_extent 1
+
+dent_len=255
+
+echo "* Create directory entries"
+
+testdir=$SCRATCH_MNT/testdir
+mkdir $testdir
+
+nr_dents=$((dbsize * 20 / dent_len))
+for i in $(seq 1 $nr_dents); do
+	dentry="$(printf "%0255d" $i)"
+	touch ${testdir}/$dentry >> $seqres.full 2>&1 || break
+done
+
+echo "Verify directory's extent count"
+nextents=$($XFS_IO_PROG -c 'stat' $testdir | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm -rf $testdir
+
+echo "* Rename: Populate destination directory"
+
+dstdir=$SCRATCH_MNT/dstdir
+mkdir $dstdir
+
+nr_dents=$((dirbsize * 20 / dent_len))
+
+echo "Populate \$dstdir by moving new directory entries"
+for i in $(seq 1 $nr_dents); do
+	dentry="$(printf "%0255d" $i)"
+	dentry=${SCRATCH_MNT}/${dentry}
+	touch $dentry || break
+	mv $dentry $dstdir >> $seqres.full 2>&1 || break
+done
+
+rm $dentry
+
+echo "Verify \$dstdir's extent count"
+
+nextents=$($XFS_IO_PROG -c 'stat' $dstdir | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm -rf $dstdir
+
+echo "* Create multiple hard links to a single file"
+
+testdir=$SCRATCH_MNT/testdir
+mkdir $testdir
+
+testfile=$SCRATCH_MNT/testfile
+touch $testfile
+
+nr_dents=$((dirbsize * 20 / dent_len))
+
+echo "Create multiple hardlinks"
+for i in $(seq 1 $nr_dents); do
+	dentry="$(printf "%0255d" $i)"
+	ln $testfile ${testdir}/${dentry} >> $seqres.full 2>&1 || break
+done
+
+rm $testfile
+
+echo "Verify directory's extent count"
+nextents=$($XFS_IO_PROG -c 'stat' $testdir | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm -rf $testdir
+
+echo "* Create multiple symbolic links to a single file"
+
+testdir=$SCRATCH_MNT/testdir
+mkdir $testdir
+
+testfile=$SCRATCH_MNT/testfile
+touch $testfile
+
+nr_dents=$((dirbsize * 20 / dent_len))
+
+echo "Create multiple symbolic links"
+for i in $(seq 1 $nr_dents); do
+	dentry="$(printf "%0255d" $i)"
+	ln -s $testfile ${testdir}/${dentry} >> $seqres.full 2>&1 || break;
+done
+
+rm $testfile
+
+echo "Verify directory's extent count"
+nextents=$($XFS_IO_PROG -c 'stat' $testdir | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm -rf $testdir
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/526.out b/tests/xfs/526.out
new file mode 100644
index 00000000..d055f56d
--- /dev/null
+++ b/tests/xfs/526.out
@@ -0,0 +1,17 @@
+QA output created by 526
+Format and mount fs
+Consume free space
+Create fragmented filesystem
+Inject reduce_max_iextents error tag
+Inject bmap_alloc_minlen_extent error tag
+* Create directory entries
+Verify directory's extent count
+* Rename: Populate destination directory
+Populate $dstdir by moving new directory entries
+Verify $dstdir's extent count
+* Create multiple hard links to a single file
+Create multiple hardlinks
+Verify directory's extent count
+* Create multiple symbolic links to a single file
+Create multiple symbolic links
+Verify directory's extent count
diff --git a/tests/xfs/group b/tests/xfs/group
index bd38aff0..d089797b 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -523,3 +523,4 @@
 523 auto quick realtime growfs
 524 auto quick punch zero insert collapse
 525 auto quick attr
+526 auto quick dir hardlink symlink
-- 
2.29.2

[PATCH V4 07/11] xfs: Check for extent overflow when writing to unwritten extent

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when writing to an unwritten extent.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/527     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/527.out | 11 ++++++
 tests/xfs/group   |  1 +
 3 files changed, 101 insertions(+)
 create mode 100755 tests/xfs/527
 create mode 100644 tests/xfs/527.out

diff --git a/tests/xfs/527 b/tests/xfs/527
new file mode 100755
index 00000000..cd67bce4
--- /dev/null
+++ b/tests/xfs/527
@@ -0,0 +1,89 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 527
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# writing to an unwritten extent.
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/inject
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_xfs_debug
+_require_xfs_io_command "falloc"
+_require_xfs_io_error_injection "reduce_max_iextents"
+
+echo "Format and mount fs"
+_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+
+testfile=${SCRATCH_MNT}/testfile
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+nr_blks=15
+
+for io in Buffered Direct; do
+	echo "* $io write to unwritten extent"
+
+	echo "Fallocate $nr_blks blocks"
+	$XFS_IO_PROG -f -c "falloc 0 $((nr_blks * bsize))" $testfile >> $seqres.full
+
+	if [[ $io == "Buffered" ]]; then
+		xfs_io_flag=""
+	else
+		xfs_io_flag="-d"
+	fi
+
+	echo "$io write to every other block of fallocated space"
+	for i in $(seq 1 2 $((nr_blks - 1))); do
+		$XFS_IO_PROG -f -s $xfs_io_flag -c "pwrite $((i * bsize)) $bsize" \
+		       $testfile >> $seqres.full 2>&1
+		[[ $? != 0 ]] && break
+	done
+
+	echo "Verify \$testfile's extent count"
+	nextents=$($XFS_IO_PROG -c 'stat' $testfile | grep nextents)
+	nextents=${nextents##fsxattr.nextents = }
+	if (( $nextents > 10 )); then
+		echo "Extent count overflow check failed: nextents = $nextents"
+		exit 1
+	fi
+
+	rm $testfile
+done
+
+# super_block->s_wb_err will have a newer seq value when compared to "/"'s
+# file->f_sb_err. Consume it here so that xfs_scrub can does not error out.
+$XFS_IO_PROG -c syncfs $SCRATCH_MNT >> $seqres.full 2>&1
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/527.out b/tests/xfs/527.out
new file mode 100644
index 00000000..3597ad92
--- /dev/null
+++ b/tests/xfs/527.out
@@ -0,0 +1,11 @@
+QA output created by 527
+Format and mount fs
+Inject reduce_max_iextents error tag
+* Buffered write to unwritten extent
+Fallocate 15 blocks
+Buffered write to every other block of fallocated space
+Verify $testfile's extent count
+* Direct write to unwritten extent
+Fallocate 15 blocks
+Direct write to every other block of fallocated space
+Verify $testfile's extent count
diff --git a/tests/xfs/group b/tests/xfs/group
index d089797b..627813fe 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -524,3 +524,4 @@
 524 auto quick punch zero insert collapse
 525 auto quick attr
 526 auto quick dir hardlink symlink
+527 auto quick
-- 
2.29.2

[PATCH V4 08/11] xfs: Check for extent overflow when moving extent from cow to data fork

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when writing to/funshare-ing a shared extent.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/528     | 110 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/528.out |  12 +++++
 tests/xfs/group   |   1 +
 3 files changed, 123 insertions(+)
 create mode 100755 tests/xfs/528
 create mode 100644 tests/xfs/528.out

diff --git a/tests/xfs/528 b/tests/xfs/528
new file mode 100755
index 00000000..269d368d
--- /dev/null
+++ b/tests/xfs/528
@@ -0,0 +1,110 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 528
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# writing to a shared extent.
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+. ./common/inject
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_scratch_reflink
+_require_xfs_debug
+_require_xfs_io_command "reflink"
+_require_xfs_io_command "funshare"
+_require_xfs_io_error_injection "reduce_max_iextents"
+
+echo "Format and mount fs"
+_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_block_size $SCRATCH_MNT)
+
+nr_blks=15
+
+srcfile=${SCRATCH_MNT}/srcfile
+dstfile=${SCRATCH_MNT}/dstfile
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Create a \$srcfile having an extent of length $nr_blks blocks"
+$XFS_IO_PROG -f -c "pwrite -b $((nr_blks * bsize))  0 $((nr_blks * bsize))" \
+       -c fsync $srcfile  >> $seqres.full
+
+echo "* Write to shared extent"
+
+echo "Share the extent with \$dstfile"
+$XFS_IO_PROG -f -c "reflink $srcfile" $dstfile >> $seqres.full
+
+echo "Buffered write to every other block of \$dstfile's shared extent"
+for i in $(seq 1 2 $((nr_blks - 1))); do
+	$XFS_IO_PROG -f -s -c "pwrite $((i * bsize)) $bsize" $dstfile \
+	       >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+echo "Verify \$dstfile's extent count"
+nextents=$($XFS_IO_PROG -c 'stat' $dstfile | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+rm $dstfile
+
+echo "* Funshare shared extent"
+
+echo "Share the extent with \$dstfile"
+$XFS_IO_PROG -f -c "reflink $srcfile" $dstfile >> $seqres.full
+
+echo "Funshare every other block of \$dstfile's shared extent"
+for i in $(seq 1 2 $((nr_blks - 1))); do
+	$XFS_IO_PROG -f -s -c "funshare $((i * bsize)) $bsize" $dstfile \
+	       >> $seqres.full 2>&1
+	[[ $? != 0 ]] && break
+done
+
+echo "Verify \$dstfile's extent count"
+nextents=$($XFS_IO_PROG -c 'stat' $dstfile | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+# super_block->s_wb_err will have a newer seq value when compared to "/"'s
+# file->f_sb_err. Consume it here so that xfs_scrub can does not error out.
+$XFS_IO_PROG -c syncfs $SCRATCH_MNT >> $seqres.full 2>&1
+
+# success, all done
+status=0
+exit
+ 
diff --git a/tests/xfs/528.out b/tests/xfs/528.out
new file mode 100644
index 00000000..d0f2c878
--- /dev/null
+++ b/tests/xfs/528.out
@@ -0,0 +1,12 @@
+QA output created by 528
+Format and mount fs
+Inject reduce_max_iextents error tag
+Create a $srcfile having an extent of length 15 blocks
+* Write to shared extent
+Share the extent with $dstfile
+Buffered write to every other block of $dstfile's shared extent
+Verify $dstfile's extent count
+* Funshare shared extent
+Share the extent with $dstfile
+Funshare every other block of $dstfile's shared extent
+Verify $dstfile's extent count
diff --git a/tests/xfs/group b/tests/xfs/group
index 627813fe..c85aac6b 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -525,3 +525,4 @@
 525 auto quick attr
 526 auto quick dir hardlink symlink
 527 auto quick
+528 auto quick reflink
-- 
2.29.2

[PATCH V4 09/11] xfs: Check for extent overflow when remapping an extent

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when remapping extents from one file's inode fork to another.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/529     | 82 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/529.out |  8 +++++
 tests/xfs/group   |  1 +
 3 files changed, 91 insertions(+)
 create mode 100755 tests/xfs/529
 create mode 100644 tests/xfs/529.out

diff --git a/tests/xfs/529 b/tests/xfs/529
new file mode 100755
index 00000000..f6a5922f
--- /dev/null
+++ b/tests/xfs/529
@@ -0,0 +1,82 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 529
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# remapping extents from one file's inode fork to another.
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+. ./common/inject
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_scratch_reflink
+_require_xfs_debug
+_require_xfs_io_command "reflink"
+_require_xfs_io_error_injection "reduce_max_iextents"
+
+echo "* Reflink remap extents"
+
+echo "Format and mount fs"
+_scratch_mkfs >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_block_size $SCRATCH_MNT)
+
+srcfile=${SCRATCH_MNT}/srcfile
+dstfile=${SCRATCH_MNT}/dstfile
+
+nr_blks=15
+
+echo "Create \$srcfile having an extent of length $nr_blks blocks"
+$XFS_IO_PROG -f -c "pwrite -b $((nr_blks * bsize))  0 $((nr_blks * bsize))" \
+       -c fsync $srcfile >> $seqres.full
+
+echo "Create \$dstfile having an extent of length $nr_blks blocks"
+$XFS_IO_PROG -f -c "pwrite -b $((nr_blks * bsize))  0 $((nr_blks * bsize))" \
+       -c fsync $dstfile >> $seqres.full
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Reflink every other block from \$srcfile into \$dstfile"
+for i in $(seq 1 2 $((nr_blks - 1))); do
+	$XFS_IO_PROG -f -c "reflink $srcfile $((i * bsize)) $((i * bsize)) $bsize" \
+	       $dstfile >> $seqres.full 2>&1
+done
+
+echo "Verify \$dstfile's extent count"
+nextents=$($XFS_IO_PROG -c 'stat' $dstfile | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+if (( $nextents > 10 )); then
+	echo "Extent count overflow check failed: nextents = $nextents"
+	exit 1
+fi
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/529.out b/tests/xfs/529.out
new file mode 100644
index 00000000..687a8bd2
--- /dev/null
+++ b/tests/xfs/529.out
@@ -0,0 +1,8 @@
+QA output created by 529
+* Reflink remap extents
+Format and mount fs
+Create $srcfile having an extent of length 15 blocks
+Create $dstfile having an extent of length 15 blocks
+Inject reduce_max_iextents error tag
+Reflink every other block from $srcfile into $dstfile
+Verify $dstfile's extent count
diff --git a/tests/xfs/group b/tests/xfs/group
index c85aac6b..bc3958b3 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -526,3 +526,4 @@
 526 auto quick dir hardlink symlink
 527 auto quick
 528 auto quick reflink
+529 auto quick reflink
-- 
2.29.2

[PATCH V4 10/11] xfs: Check for extent overflow when swapping extents

[Chandan Babu R]

This test verifies that XFS does not cause inode fork's extent count to
overflow when swapping forks across two files.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/530     | 109 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/530.out |  13 ++++++
 tests/xfs/group   |   1 +
 3 files changed, 123 insertions(+)
 create mode 100755 tests/xfs/530
 create mode 100644 tests/xfs/530.out

diff --git a/tests/xfs/530 b/tests/xfs/530
new file mode 100755
index 00000000..0986d8bf
--- /dev/null
+++ b/tests/xfs/530
@@ -0,0 +1,109 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 530
+#
+# Verify that XFS does not cause inode fork's extent count to overflow when
+# swapping forks between files
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/inject
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_xfs_debug
+_require_xfs_scratch_rmapbt
+_require_xfs_io_command "fcollapse"
+_require_xfs_io_command "swapext"
+_require_xfs_io_error_injection "reduce_max_iextents"
+
+echo "* Swap extent forks"
+
+echo "Format and mount fs"
+_scratch_mkfs >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_block_size $SCRATCH_MNT)
+
+srcfile=${SCRATCH_MNT}/srcfile
+donorfile=${SCRATCH_MNT}/donorfile
+
+echo "Create \$donorfile having an extent of length 67 blocks"
+$XFS_IO_PROG -f -s -c "pwrite -b $((17 * bsize)) 0 $((17 * bsize))" $donorfile \
+       >> $seqres.full
+
+# After the for loop the donor file will have the following extent layout
+# | 0-4 | 5 | 6 | 7 | 8 | 9 | 10 |
+echo "Fragment \$donorfile"
+for i in $(seq 5 10); do
+	start_offset=$((i * bsize))
+	$XFS_IO_PROG -f -c "fcollapse $start_offset $bsize" $donorfile >> $seqres.full
+done
+
+echo "Create \$srcfile having an extent of length 18 blocks"
+$XFS_IO_PROG -f -s -c "pwrite -b $((18 * bsize)) 0 $((18 * bsize))" $srcfile \
+       >> $seqres.full
+
+echo "Fragment \$srcfile"
+# After the for loop the src file will have the following extent layout
+# | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7-10 |
+for i in $(seq 1 7); do
+	start_offset=$((i * bsize))
+	$XFS_IO_PROG -f -c "fcollapse $start_offset $bsize" $srcfile >> $seqres.full
+done
+
+echo "Collect \$donorfile's extent count"
+donor_nr_exts=$($XFS_IO_PROG -c 'stat' $donorfile | grep nextents)
+donor_nr_exts=${donor_nr_exts##fsxattr.nextents = }
+
+echo "Collect \$srcfile's extent count"
+src_nr_exts=$($XFS_IO_PROG -c 'stat' $srcfile | grep nextents)
+src_nr_exts=${src_nr_exts##fsxattr.nextents = }
+
+echo "Inject reduce_max_iextents error tag"
+_scratch_inject_error reduce_max_iextents 1
+
+echo "Swap \$srcfile's and \$donorfile's extent forks"
+$XFS_IO_PROG -f -c "swapext $donorfile" $srcfile >> $seqres.full 2>&1
+
+echo "Check for \$donorfile's extent count overflow"
+nextents=$($XFS_IO_PROG -c 'stat' $donorfile | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+
+if (( $nextents == $src_nr_exts )); then
+	echo "\$donorfile: Extent count overflow check failed"
+fi
+
+echo "Check for \$srcfile's extent count overflow"
+nextents=$($XFS_IO_PROG -c 'stat' $srcfile | grep nextents)
+nextents=${nextents##fsxattr.nextents = }
+
+if (( $nextents == $donor_nr_exts )); then
+	echo "\$srcfile: Extent count overflow check failed"
+fi
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/530.out b/tests/xfs/530.out
new file mode 100644
index 00000000..9f55608b
--- /dev/null
+++ b/tests/xfs/530.out
@@ -0,0 +1,13 @@
+QA output created by 530
+* Swap extent forks
+Format and mount fs
+Create $donorfile having an extent of length 67 blocks
+Fragment $donorfile
+Create $srcfile having an extent of length 18 blocks
+Fragment $srcfile
+Collect $donorfile's extent count
+Collect $srcfile's extent count
+Inject reduce_max_iextents error tag
+Swap $srcfile's and $donorfile's extent forks
+Check for $donorfile's extent count overflow
+Check for $srcfile's extent count overflow
diff --git a/tests/xfs/group b/tests/xfs/group
index bc3958b3..81a15582 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -527,3 +527,4 @@
 527 auto quick
 528 auto quick reflink
 529 auto quick reflink
+530 auto quick
-- 
2.29.2

[PATCH V4 11/11] xfs: Stress test with bmap_alloc_minlen_extent error tag enabled

[Chandan Babu R]

This commit adds a stress test that executes fsstress with
bmap_alloc_minlen_extent error tag enabled.

Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/531     | 84 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/531.out |  7 ++++
 tests/xfs/group   |  1 +
 3 files changed, 92 insertions(+)
 create mode 100755 tests/xfs/531
 create mode 100644 tests/xfs/531.out

diff --git a/tests/xfs/531 b/tests/xfs/531
new file mode 100755
index 00000000..fd92c3ea
--- /dev/null
+++ b/tests/xfs/531
@@ -0,0 +1,84 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
+#
+# FS QA Test 531
+#
+# Execute fsstress with bmap_alloc_minlen_extent error tag enabled.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/inject
+. ./common/populate
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs xfs
+_require_scratch
+_require_xfs_debug
+_require_test_program "punch-alternating"
+_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
+
+echo "Format and mount fs"
+_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
+_scratch_mount >> $seqres.full
+
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+
+echo "Consume free space"
+fillerdir=$SCRATCH_MNT/fillerdir
+nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
+nr_free_blks=$((nr_free_blks * 90 / 100))
+
+_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
+
+echo "Create fragmented filesystem"
+for dentry in $(ls -1 $fillerdir/); do
+	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
+done
+
+echo "Inject bmap_alloc_minlen_extent error tag"
+_scratch_inject_error bmap_alloc_minlen_extent 1
+
+echo "Scale fsstress args"
+args=$(_scale_fsstress_args -p $((LOAD_FACTOR * 75)) -n $((TIME_FACTOR * 1000)))
+
+echo "Execute fsstress in background"
+$FSSTRESS_PROG -d $SCRATCH_MNT $args \
+		 -f bulkstat=0 \
+		 -f bulkstat1=0 \
+		 -f fiemap=0 \
+		 -f getattr=0 \
+		 -f getdents=0 \
+		 -f getfattr=0 \
+		 -f listfattr=0 \
+		 -f mread=0 \
+		 -f read=0 \
+		 -f readlink=0 \
+		 -f readv=0 \
+		 -f stat=0 \
+		 -f aread=0 \
+		 -f dread=0 > /dev/null 2>&1
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/531.out b/tests/xfs/531.out
new file mode 100644
index 00000000..67f40654
--- /dev/null
+++ b/tests/xfs/531.out
@@ -0,0 +1,7 @@
+QA output created by 531
+Format and mount fs
+Consume free space
+Create fragmented filesystem
+Inject bmap_alloc_minlen_extent error tag
+Scale fsstress args
+Execute fsstress in background
diff --git a/tests/xfs/group b/tests/xfs/group
index 81a15582..f4cb5af6 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -528,3 +528,4 @@
 528 auto quick reflink
 529 auto quick reflink
 530 auto quick
+531 auto stress
-- 
2.29.2

Re: [PATCH V4 01/11] common/xfs: Add a helper to get an inode fork's extent count

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:12AM +0530, Chandan Babu R wrote:
> This commit adds the helper _scratch_get_iext_count() which returns an
> inode fork's extent count.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  common/xfs | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/common/xfs b/common/xfs
> index 3f5c14ba..641d6195 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -903,6 +903,28 @@ _scratch_get_bmx_prefix() {
>  	return 1
>  }
>  
> +_scratch_get_iext_count()
> +{
> +	ino=$1
> +	whichfork=$2

Function variables should be declared with 'local' so they don't bleed
into the global namespace (yay bash!), e.g.

	local ino="$1"

Also, now that Eric has landed the xfs_db 'path' command upstream, you
might consider using it:

	_scratch_xfs_get_metadata_field "core.nextents" "path /windows/system.ini"

> +
> +	case $whichfork in
> +		"attr")
> +			field=core.naextents
> +			;;
> +		"data")
> +			field=core.nextents
> +			;;
> +		*)
> +			return 1
> +	esac
> +
> +	nextents=$(_scratch_xfs_db  -c "inode $ino" -c "print $field")
> +	nextents=${nextents##${field} = }

_scratch_xfs_get_metadata_field?

--D

> +
> +	echo $nextents
> +}
> +
>  #
>  # Ensures that we don't pass any mount options incompatible with XFS v4
>  #
> -- 
> 2.29.2
> 

Re: [PATCH V4 02/11] xfs: Check for extent overflow when trivally adding a new extent

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:13AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when adding a single extent while there's no possibility of
> splitting an existing mapping.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/522     | 173 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/522.out |  20 ++++++
>  tests/xfs/group   |   1 +
>  3 files changed, 194 insertions(+)
>  create mode 100755 tests/xfs/522
>  create mode 100644 tests/xfs/522.out
> 
> diff --git a/tests/xfs/522 b/tests/xfs/522
> new file mode 100755
> index 00000000..33f0591e
> --- /dev/null
> +++ b/tests/xfs/522
> @@ -0,0 +1,173 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 522
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# adding a single extent while there's no possibility of splitting an existing
> +# mapping.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/quota
> +. ./common/inject
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_quota
> +_require_xfs_debug
> +_require_test_program "punch-alternating"
> +_require_xfs_io_command "falloc"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full
> +_scratch_mount -o uquota >> $seqres.full
> +
> +bsize=$(_get_file_block_size $SCRATCH_MNT)
> +
> +echo "* Delalloc to written extent conversion"

What happens if delalloc is disabled?  e.g. the mkfs parameters have
extent size hints set on the root directory?

(I think it'll be fine since you write every other block...)

> +testfile=$SCRATCH_MNT/testfile
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +nr_blks=$((15 * 2))
> +
> +echo "Create fragmented file"
> +for i in $(seq 0 2 $((nr_blks - 1))); do
> +	$XFS_IO_PROG -f -s -c "pwrite $((i * bsize)) $bsize" $testfile \
> +	       >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +echo "Verify \$testfile's extent count"
> +
> +nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
> +nextents=${nextents##fsxattr.nextents = }

/me wonders if it's time for a helper to extract these fields...

> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm $testfile
> +
> +echo "* Fallocate unwritten extents"
> +
> +echo "Fallocate fragmented file"
> +for i in $(seq 0 2 $((nr_blks - 1))); do
> +	$XFS_IO_PROG -f -c "falloc $((i * bsize)) $bsize" $testfile \
> +	       >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +echo "Verify \$testfile's extent count"
> +
> +nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm $testfile
> +
> +echo "* Directio write"
> +
> +echo "Create fragmented file via directio writes"
> +for i in $(seq 0 2 $((nr_blks - 1))); do
> +	$XFS_IO_PROG -d -s -f -c "pwrite $((i * bsize)) $bsize" $testfile \
> +	       >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +echo "Verify \$testfile's extent count"
> +
> +nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm $testfile
> +
> +echo "* Extend quota inodes"
> +
> +echo "Disable reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 0
> +
> +echo "Consume free space"
> +fillerdir=$SCRATCH_MNT/fillerdir
> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> +nr_free_blks=$((nr_free_blks * 90 / 100))
> +
> +_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
> +
> +echo "Create fragmented filesystem"
> +for dentry in $(ls -1 $fillerdir/); do
> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> +done
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Inject bmap_alloc_minlen_extent error tag"
> +_scratch_inject_error bmap_alloc_minlen_extent 1

Are we testing to see if the fs blows up when the quota inodes hit max
iext count?  If so, please put that in the comments for this section.

> +
> +nr_blks=20
> +
> +# This is a rough calculation; It doesn't take block headers into
> +# consideration.
> +# gdb -batch vmlinux -ex 'print sizeof(struct xfs_dqblk)'
> +# $1 = 136
> +nr_quotas_per_block=$((bsize / 136))
> +nr_quotas=$((nr_quotas_per_block * nr_blks))
> +
> +echo "Extend uquota file"
> +for i in $(seq 0 $nr_quotas_per_block $nr_quotas); do
> +	chown $i $testfile >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +_scratch_unmount >> $seqres.full
> +
> +echo "Verify uquota inode's extent count"
> +uquotino=$(_scratch_xfs_db -c sb -c "print uquotino")
> +uquotino=${uquotino##uquotino = }

_scratch_xfs_get_metadata_field

> +
> +nextents=$(_scratch_get_iext_count $uquotino data || \
> +		   _fail "Unable to obtain inode fork's extent count")
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/522.out b/tests/xfs/522.out
> new file mode 100644
> index 00000000..debeb004
> --- /dev/null
> +++ b/tests/xfs/522.out
> @@ -0,0 +1,20 @@
> +QA output created by 522
> +Format and mount fs
> +* Delalloc to written extent conversion
> +Inject reduce_max_iextents error tag
> +Create fragmented file
> +Verify $testfile's extent count
> +* Fallocate unwritten extents
> +Fallocate fragmented file
> +Verify $testfile's extent count
> +* Directio write
> +Create fragmented file via directio writes
> +Verify $testfile's extent count
> +* Extend quota inodes
> +Disable reduce_max_iextents error tag
> +Consume free space
> +Create fragmented filesystem
> +Inject reduce_max_iextents error tag
> +Inject bmap_alloc_minlen_extent error tag
> +Extend uquota file
> +Verify uquota inode's extent count
> diff --git a/tests/xfs/group b/tests/xfs/group
> index b89c0a4e..1831f0b5 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -519,3 +519,4 @@
>  519 auto quick reflink
>  520 auto quick reflink
>  521 auto quick realtime growfs
> +522 auto quick quota
> -- 
> 2.29.2
> 

Re: [PATCH V4 03/11] xfs: Check for extent overflow when growing realtime bitmap/summary inodes

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:14AM +0530, Chandan Babu R wrote:
> Verify that XFS does not cause realtime bitmap/summary inode fork's
> extent count to overflow when growing the realtime volume associated
> with a filesystem.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/523     | 119 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/523.out |  11 +++++
>  tests/xfs/group   |   1 +
>  3 files changed, 131 insertions(+)
>  create mode 100755 tests/xfs/523
>  create mode 100644 tests/xfs/523.out
> 
> diff --git a/tests/xfs/523 b/tests/xfs/523
> new file mode 100755
> index 00000000..145f0ff6
> --- /dev/null
> +++ b/tests/xfs/523
> @@ -0,0 +1,119 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 523
> +#
> +# Verify that XFS does not cause bitmap/summary inode fork's extent count to
> +# overflow when growing an the realtime volume of the filesystem.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	_scratch_unmount >> $seqres.full 2>&1
> +	test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1
> +	rm -f $tmp.* $TEST_DIR/$seq.rtvol
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_test
> +_require_xfs_debug
> +_require_test_program "punch-alternating"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> +_require_scratch_nocheck

Might want a comment about why this test doesn't _require_realtime.

Hm.  What /does/ happen if the kernel xfs driver wasn't built with
realtime support?  I'll investigate and report back.

> +
> +echo "* Test extending rt inodes"
> +
> +_scratch_mkfs | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
> +. $tmp.mkfs
> +
> +echo "Create fake rt volume"
> +nr_bitmap_blks=25
> +nr_bits=$((nr_bitmap_blks * dbsize * 8))
> +rtextsz=$dbsize
> +rtdevsz=$((nr_bits * rtextsz))
> +truncate -s $rtdevsz $TEST_DIR/$seq.rtvol
> +rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol)
> +
> +echo "Format and mount rt volume"
> +
> +export USE_EXTERNAL=yes
> +export SCRATCH_RTDEV=$rtdev
> +_scratch_mkfs -d size=$((1024 * 1024 * 1024)) \
> +	      -r size=2M,extsize=${rtextsz} >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +echo "Consume free space"
> +fillerdir=$SCRATCH_MNT/fillerdir
> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> +nr_free_blks=$((nr_free_blks * 90 / 100))
> +
> +_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
> +
> +echo "Create fragmented filesystem"
> +for dentry in $(ls -1 $fillerdir/); do
> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> +done
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Inject bmap_alloc_minlen_extent error tag"
> +_scratch_inject_error bmap_alloc_minlen_extent 1
> +
> +echo "Grow realtime volume"
> +$XFS_GROWFS_PROG -r $SCRATCH_MNT >> $seqres.full 2>&1
> +if [[ $? == 0 ]]; then
> +	echo "Growfs succeeded; should have failed."
> +	exit 1
> +fi
> +
> +_scratch_unmount >> $seqres.full
> +
> +echo "Verify rbmino's and rsumino's extent count"
> +for rtino in rbmino rsumino; do
> +	ino=$(_scratch_xfs_db -c sb -c "print $rtino")
> +	ino=${ino##${rtino} = }
> +	echo "$rtino = $ino" >> $seqres.full
> +
> +	nextents=$(_scratch_get_iext_count $ino data || \
> +			_fail "Unable to obtain inode fork's extent count")
> +	if (( $nextents > 10 )); then
> +		echo "Extent count overflow check failed: nextents = $nextents"
> +		exit 1
> +	fi
> +done
> +
> +echo "Check filesystem"
> +_check_xfs_filesystem $SCRATCH_DEV none $rtdev
> +
> +losetup -d $rtdev
> +rm -f $TEST_DIR/$seq.rtvol
> +
> +export USE_EXTERNAL=""
> +export SCRATCH_RTDEV=""

No need to clear these, but with those two nits fixed, this looks all right.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>

--D

> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/523.out b/tests/xfs/523.out
> new file mode 100644
> index 00000000..7df02970
> --- /dev/null
> +++ b/tests/xfs/523.out
> @@ -0,0 +1,11 @@
> +QA output created by 523
> +* Test extending rt inodes
> +Create fake rt volume
> +Format and mount rt volume
> +Consume free space
> +Create fragmented filesystem
> +Inject reduce_max_iextents error tag
> +Inject bmap_alloc_minlen_extent error tag
> +Grow realtime volume
> +Verify rbmino's and rsumino's extent count
> +Check filesystem
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 1831f0b5..018c70ef 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -520,3 +520,4 @@
>  520 auto quick reflink
>  521 auto quick realtime growfs
>  522 auto quick quota
> +523 auto quick realtime growfs
> -- 
> 2.29.2
> 

Re: [PATCH V4 04/11] xfs: Check for extent overflow when punching a hole

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:15AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when punching out an extent.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/524     | 84 +++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/524.out | 19 +++++++++++
>  tests/xfs/group   |  1 +
>  3 files changed, 104 insertions(+)
>  create mode 100755 tests/xfs/524
>  create mode 100644 tests/xfs/524.out
> 
> diff --git a/tests/xfs/524 b/tests/xfs/524
> new file mode 100755
> index 00000000..79fa31d8
> --- /dev/null
> +++ b/tests/xfs/524
> @@ -0,0 +1,84 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 524
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# punching out an extent.
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_debug
> +_require_xfs_io_command "fpunch"
> +_require_xfs_io_command "finsert"
> +_require_xfs_io_command "fcollapse"
> +_require_xfs_io_command "fzero"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_file_block_size $SCRATCH_MNT)
> +nr_blks=30
> +
> +testfile=$SCRATCH_MNT/testfile
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +for op in fpunch finsert fcollapse fzero; do
> +	echo "* $op regular file"
> +
> +	echo "Create \$testfile"
> +	$XFS_IO_PROG -f -s \
> +		     -c "pwrite -b $((nr_blks * bsize)) 0 $((nr_blks * bsize))" \
> +		     $testfile  >> $seqres.full
> +
> +	echo "$op alternating blocks"
> +	for i in $(seq 1 2 $((nr_blks - 1))); do
> +		$XFS_IO_PROG -f -c "$op $((i * bsize)) $bsize" $testfile \
> +		       >> $seqres.full 2>&1
> +		[[ $? != 0 ]] && break
> +	done
> +
> +	echo "Verify \$testfile's extent count"
> +
> +	nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
> +	nextents=${nextents##fsxattr.nextents = }

Modulo my comment about refactoring fsgetxattr output into a common
helper, the rest looks ok:

Reviewed-by: Darrick J. Wong <djwong@kernel.org>

--D

> +	if (( $nextents > 10 )); then
> +		echo "Extent count overflow check failed: nextents = $nextents"
> +		exit 1
> +	fi
> +
> +	rm $testfile
> +done
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/524.out b/tests/xfs/524.out
> new file mode 100644
> index 00000000..a957f9c7
> --- /dev/null
> +++ b/tests/xfs/524.out
> @@ -0,0 +1,19 @@
> +QA output created by 524
> +Format and mount fs
> +Inject reduce_max_iextents error tag
> +* fpunch regular file
> +Create $testfile
> +fpunch alternating blocks
> +Verify $testfile's extent count
> +* finsert regular file
> +Create $testfile
> +finsert alternating blocks
> +Verify $testfile's extent count
> +* fcollapse regular file
> +Create $testfile
> +fcollapse alternating blocks
> +Verify $testfile's extent count
> +* fzero regular file
> +Create $testfile
> +fzero alternating blocks
> +Verify $testfile's extent count
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 018c70ef..3fa38c36 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -521,3 +521,4 @@
>  521 auto quick realtime growfs
>  522 auto quick quota
>  523 auto quick realtime growfs
> +524 auto quick punch zero insert collapse
> -- 
> 2.29.2
> 

Re: [PATCH V4 05/11] xfs: Check for extent overflow when adding/removing xattrs

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:16AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when adding/removing xattrs.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/525     | 141 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/525.out |  18 ++++++
>  tests/xfs/group   |   1 +
>  3 files changed, 160 insertions(+)
>  create mode 100755 tests/xfs/525
>  create mode 100644 tests/xfs/525.out
> 
> diff --git a/tests/xfs/525 b/tests/xfs/525
> new file mode 100755
> index 00000000..bdca846d
> --- /dev/null
> +++ b/tests/xfs/525
> @@ -0,0 +1,141 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 525
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# Adding/removing xattrs.

Nit: 'adding' (no need to capitalize)

Reviewed-by: Darrick J. Wong <djwong@kernel.org>

--D


> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/attr
> +. ./common/inject
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_attrs
> +_require_xfs_debug
> +_require_test_program "punch-alternating"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_block_size $SCRATCH_MNT)
> +
> +attr_len=255
> +
> +testfile=$SCRATCH_MNT/testfile
> +
> +echo "Consume free space"
> +fillerdir=$SCRATCH_MNT/fillerdir
> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> +nr_free_blks=$((nr_free_blks * 90 / 100))
> +
> +_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
> +
> +echo "Create fragmented filesystem"
> +for dentry in $(ls -1 $fillerdir/); do
> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> +done
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Inject bmap_alloc_minlen_extent error tag"
> +_scratch_inject_error bmap_alloc_minlen_extent 1
> +
> +echo "* Set xattrs"
> +
> +echo "Create \$testfile"
> +touch $testfile
> +
> +echo "Create xattrs"
> +nr_attrs=$((bsize * 20 / attr_len))
> +for i in $(seq 1 $nr_attrs); do
> +	attr="$(printf "trusted.%0247d" $i)"
> +	$SETFATTR_PROG -n "$attr" $testfile >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +echo "Verify \$testfile's naextent count"
> +
> +naextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep naextents)
> +naextents=${naextents##fsxattr.naextents = }
> +if (( $naextents > 10 )); then
> +	echo "Extent count overflow check failed: naextents = $naextents"
> +	exit 1
> +fi
> +
> +echo "Remove \$testfile"
> +rm $testfile
> +
> +echo "* Remove xattrs"
> +
> +echo "Create \$testfile"
> +touch $testfile
> +
> +echo "Disable reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 0
> +
> +echo "Create initial xattr extents"
> +
> +naextents=0
> +last=""
> +start=1
> +nr_attrs=$((bsize / attr_len))
> +
> +while (( $naextents < 4 )); do
> +	end=$((start + nr_attrs - 1))
> +
> +	for i in $(seq $start $end); do
> +		attr="$(printf "trusted.%0247d" $i)"
> +		$SETFATTR_PROG -n $attr $testfile
> +	done
> +
> +	start=$((end + 1))
> +
> +	naextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep naextents)
> +	naextents=${naextents##fsxattr.naextents = }
> +done
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Remove xattr to trigger -EFBIG"
> +attr="$(printf "trusted.%0247d" 1)"
> +$SETFATTR_PROG -x "$attr" $testfile >> $seqres.full 2>&1
> +if [[ $? == 0 ]]; then
> +	echo "Xattr removal succeeded; Should have failed "
> +	exit 1
> +fi
> +
> +rm $testfile && echo "Successfully removed \$testfile"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/525.out b/tests/xfs/525.out
> new file mode 100644
> index 00000000..74b152d9
> --- /dev/null
> +++ b/tests/xfs/525.out
> @@ -0,0 +1,18 @@
> +QA output created by 525
> +Format and mount fs
> +Consume free space
> +Create fragmented filesystem
> +Inject reduce_max_iextents error tag
> +Inject bmap_alloc_minlen_extent error tag
> +* Set xattrs
> +Create $testfile
> +Create xattrs
> +Verify $testfile's naextent count
> +Remove $testfile
> +* Remove xattrs
> +Create $testfile
> +Disable reduce_max_iextents error tag
> +Create initial xattr extents
> +Inject reduce_max_iextents error tag
> +Remove xattr to trigger -EFBIG
> +Successfully removed $testfile
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 3fa38c36..bd38aff0 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -522,3 +522,4 @@
>  522 auto quick quota
>  523 auto quick realtime growfs
>  524 auto quick punch zero insert collapse
> +525 auto quick attr
> -- 
> 2.29.2
> 

Re: [PATCH V4 06/11] xfs: Check for extent overflow when adding/removing dir entries

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:17AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when adding/removing directory entries.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/526     | 186 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/526.out |  17 +++++
>  tests/xfs/group   |   1 +
>  3 files changed, 204 insertions(+)
>  create mode 100755 tests/xfs/526
>  create mode 100644 tests/xfs/526.out
> 
> diff --git a/tests/xfs/526 b/tests/xfs/526
> new file mode 100755
> index 00000000..5a789d61
> --- /dev/null
> +++ b/tests/xfs/526
> @@ -0,0 +1,186 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 526
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# adding/removing directory entries.
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_debug
> +_require_test_program "punch-alternating"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> +
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
> +. $tmp.mkfs
> +
> +# Filesystems with directory block size greater than one FSB will not be tested,
> +# since "7 (i.e. XFS_DA_NODE_MAXDEPTH + 1 data block + 1 free block) * 2 (fsb
> +# count) = 14" is greater than the pseudo max extent count limit of 10.
> +# Extending the pseudo max limit won't help either.  Consider the case where 1
> +# FSB is 1k in size and 1 dir block is 64k in size (i.e. fsb count = 64). In
> +# this case, the pseudo max limit has to be greater than 7 * 64 = 448 extents.
> +if (( $dbsize != $dirbsize )); then
> +	_notrun "FSB size ($dbsize) and directory block size ($dirbsize) do not match"
> +fi

But what about the case where fsb is 1k and dirblocks are 4k? :)

I admit I'm reacting to my expectation that we would _notrun here based
on the output of a more computation, not just fsb != dirblocksize.  But
I dunno, maybe you've already worked that out?

(The rest of the test looks good to me.)

--D

> +
> +echo "Format and mount fs"
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +echo "Consume free space"
> +fillerdir=$SCRATCH_MNT/fillerdir
> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> +nr_free_blks=$((nr_free_blks * 90 / 100))
> +
> +_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
> +
> +echo "Create fragmented filesystem"
> +for dentry in $(ls -1 $fillerdir/); do
> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> +done
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Inject bmap_alloc_minlen_extent error tag"
> +_scratch_inject_error bmap_alloc_minlen_extent 1
> +
> +dent_len=255
> +
> +echo "* Create directory entries"
> +
> +testdir=$SCRATCH_MNT/testdir
> +mkdir $testdir
> +
> +nr_dents=$((dbsize * 20 / dent_len))
> +for i in $(seq 1 $nr_dents); do
> +	dentry="$(printf "%0255d" $i)"
> +	touch ${testdir}/$dentry >> $seqres.full 2>&1 || break
> +done
> +
> +echo "Verify directory's extent count"
> +nextents=$($XFS_IO_PROG -c 'stat' $testdir | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm -rf $testdir
> +
> +echo "* Rename: Populate destination directory"
> +
> +dstdir=$SCRATCH_MNT/dstdir
> +mkdir $dstdir
> +
> +nr_dents=$((dirbsize * 20 / dent_len))
> +
> +echo "Populate \$dstdir by moving new directory entries"
> +for i in $(seq 1 $nr_dents); do
> +	dentry="$(printf "%0255d" $i)"
> +	dentry=${SCRATCH_MNT}/${dentry}
> +	touch $dentry || break
> +	mv $dentry $dstdir >> $seqres.full 2>&1 || break
> +done
> +
> +rm $dentry
> +
> +echo "Verify \$dstdir's extent count"
> +
> +nextents=$($XFS_IO_PROG -c 'stat' $dstdir | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm -rf $dstdir
> +
> +echo "* Create multiple hard links to a single file"
> +
> +testdir=$SCRATCH_MNT/testdir
> +mkdir $testdir
> +
> +testfile=$SCRATCH_MNT/testfile
> +touch $testfile
> +
> +nr_dents=$((dirbsize * 20 / dent_len))
> +
> +echo "Create multiple hardlinks"
> +for i in $(seq 1 $nr_dents); do
> +	dentry="$(printf "%0255d" $i)"
> +	ln $testfile ${testdir}/${dentry} >> $seqres.full 2>&1 || break
> +done
> +
> +rm $testfile
> +
> +echo "Verify directory's extent count"
> +nextents=$($XFS_IO_PROG -c 'stat' $testdir | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm -rf $testdir
> +
> +echo "* Create multiple symbolic links to a single file"
> +
> +testdir=$SCRATCH_MNT/testdir
> +mkdir $testdir
> +
> +testfile=$SCRATCH_MNT/testfile
> +touch $testfile
> +
> +nr_dents=$((dirbsize * 20 / dent_len))
> +
> +echo "Create multiple symbolic links"
> +for i in $(seq 1 $nr_dents); do
> +	dentry="$(printf "%0255d" $i)"
> +	ln -s $testfile ${testdir}/${dentry} >> $seqres.full 2>&1 || break;
> +done
> +
> +rm $testfile
> +
> +echo "Verify directory's extent count"
> +nextents=$($XFS_IO_PROG -c 'stat' $testdir | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm -rf $testdir
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/526.out b/tests/xfs/526.out
> new file mode 100644
> index 00000000..d055f56d
> --- /dev/null
> +++ b/tests/xfs/526.out
> @@ -0,0 +1,17 @@
> +QA output created by 526
> +Format and mount fs
> +Consume free space
> +Create fragmented filesystem
> +Inject reduce_max_iextents error tag
> +Inject bmap_alloc_minlen_extent error tag
> +* Create directory entries
> +Verify directory's extent count
> +* Rename: Populate destination directory
> +Populate $dstdir by moving new directory entries
> +Verify $dstdir's extent count
> +* Create multiple hard links to a single file
> +Create multiple hardlinks
> +Verify directory's extent count
> +* Create multiple symbolic links to a single file
> +Create multiple symbolic links
> +Verify directory's extent count
> diff --git a/tests/xfs/group b/tests/xfs/group
> index bd38aff0..d089797b 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -523,3 +523,4 @@
>  523 auto quick realtime growfs
>  524 auto quick punch zero insert collapse
>  525 auto quick attr
> +526 auto quick dir hardlink symlink
> -- 
> 2.29.2
> 

Re: [PATCH V4 07/11] xfs: Check for extent overflow when writing to unwritten extent

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:18AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when writing to an unwritten extent.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/527     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/527.out | 11 ++++++
>  tests/xfs/group   |  1 +
>  3 files changed, 101 insertions(+)
>  create mode 100755 tests/xfs/527
>  create mode 100644 tests/xfs/527.out
> 
> diff --git a/tests/xfs/527 b/tests/xfs/527
> new file mode 100755
> index 00000000..cd67bce4
> --- /dev/null
> +++ b/tests/xfs/527
> @@ -0,0 +1,89 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 527
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# writing to an unwritten extent.
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_debug
> +_require_xfs_io_command "falloc"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_file_block_size $SCRATCH_MNT)
> +
> +testfile=${SCRATCH_MNT}/testfile
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +nr_blks=15
> +
> +for io in Buffered Direct; do

# First test buffered writes, then direct writes.
for $xfs_io_flag in "" "-d"; do ?

> +	echo "* $io write to unwritten extent"
> +
> +	echo "Fallocate $nr_blks blocks"
> +	$XFS_IO_PROG -f -c "falloc 0 $((nr_blks * bsize))" $testfile >> $seqres.full
> +
> +	if [[ $io == "Buffered" ]]; then
> +		xfs_io_flag=""
> +	else
> +		xfs_io_flag="-d"
> +	fi

...because then you can skip this part.

> +
> +	echo "$io write to every other block of fallocated space"
> +	for i in $(seq 1 2 $((nr_blks - 1))); do
> +		$XFS_IO_PROG -f -s $xfs_io_flag -c "pwrite $((i * bsize)) $bsize" \
> +		       $testfile >> $seqres.full 2>&1
> +		[[ $? != 0 ]] && break
> +	done
> +
> +	echo "Verify \$testfile's extent count"
> +	nextents=$($XFS_IO_PROG -c 'stat' $testfile | grep nextents)
> +	nextents=${nextents##fsxattr.nextents = }
> +	if (( $nextents > 10 )); then
> +		echo "Extent count overflow check failed: nextents = $nextents"
> +		exit 1
> +	fi
> +
> +	rm $testfile
> +done
> +
> +# super_block->s_wb_err will have a newer seq value when compared to "/"'s
> +# file->f_sb_err. Consume it here so that xfs_scrub can does not error out.
> +$XFS_IO_PROG -c syncfs $SCRATCH_MNT >> $seqres.full 2>&1

I wonder, should _check_xfs_filesystem should syncfs to clear old EIOs
before running xfs_scrub?  I occasionally see this pop up on
generic/204.

(Everything else here looks ok.)

--D

> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/527.out b/tests/xfs/527.out
> new file mode 100644
> index 00000000..3597ad92
> --- /dev/null
> +++ b/tests/xfs/527.out
> @@ -0,0 +1,11 @@
> +QA output created by 527
> +Format and mount fs
> +Inject reduce_max_iextents error tag
> +* Buffered write to unwritten extent
> +Fallocate 15 blocks
> +Buffered write to every other block of fallocated space
> +Verify $testfile's extent count
> +* Direct write to unwritten extent
> +Fallocate 15 blocks
> +Direct write to every other block of fallocated space
> +Verify $testfile's extent count
> diff --git a/tests/xfs/group b/tests/xfs/group
> index d089797b..627813fe 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -524,3 +524,4 @@
>  524 auto quick punch zero insert collapse
>  525 auto quick attr
>  526 auto quick dir hardlink symlink
> +527 auto quick
> -- 
> 2.29.2
> 

Re: [PATCH V4 08/11] xfs: Check for extent overflow when moving extent from cow to data fork

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:19AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when writing to/funshare-ing a shared extent.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/528     | 110 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/528.out |  12 +++++
>  tests/xfs/group   |   1 +
>  3 files changed, 123 insertions(+)
>  create mode 100755 tests/xfs/528
>  create mode 100644 tests/xfs/528.out
> 
> diff --git a/tests/xfs/528 b/tests/xfs/528
> new file mode 100755
> index 00000000..269d368d
> --- /dev/null
> +++ b/tests/xfs/528
> @@ -0,0 +1,110 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 528
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# writing to a shared extent.
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/reflink
> +. ./common/inject
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_scratch_reflink
> +_require_xfs_debug
> +_require_xfs_io_command "reflink"
> +_require_xfs_io_command "funshare"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_block_size $SCRATCH_MNT)
> +
> +nr_blks=15
> +
> +srcfile=${SCRATCH_MNT}/srcfile
> +dstfile=${SCRATCH_MNT}/dstfile
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Create a \$srcfile having an extent of length $nr_blks blocks"
> +$XFS_IO_PROG -f -c "pwrite -b $((nr_blks * bsize))  0 $((nr_blks * bsize))" \
> +       -c fsync $srcfile  >> $seqres.full
> +
> +echo "* Write to shared extent"
> +
> +echo "Share the extent with \$dstfile"
> +$XFS_IO_PROG -f -c "reflink $srcfile" $dstfile >> $seqres.full

_reflink?

> +
> +echo "Buffered write to every other block of \$dstfile's shared extent"
> +for i in $(seq 1 2 $((nr_blks - 1))); do
> +	$XFS_IO_PROG -f -s -c "pwrite $((i * bsize)) $bsize" $dstfile \
> +	       >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +echo "Verify \$dstfile's extent count"
> +nextents=$($XFS_IO_PROG -c 'stat' $dstfile | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +rm $dstfile
> +
> +echo "* Funshare shared extent"
> +
> +echo "Share the extent with \$dstfile"
> +$XFS_IO_PROG -f -c "reflink $srcfile" $dstfile >> $seqres.full
> +
> +echo "Funshare every other block of \$dstfile's shared extent"
> +for i in $(seq 1 2 $((nr_blks - 1))); do
> +	$XFS_IO_PROG -f -s -c "funshare $((i * bsize)) $bsize" $dstfile \
> +	       >> $seqres.full 2>&1
> +	[[ $? != 0 ]] && break
> +done
> +
> +echo "Verify \$dstfile's extent count"
> +nextents=$($XFS_IO_PROG -c 'stat' $dstfile | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +# super_block->s_wb_err will have a newer seq value when compared to "/"'s
> +# file->f_sb_err. Consume it here so that xfs_scrub can does not error out.
> +$XFS_IO_PROG -c syncfs $SCRATCH_MNT >> $seqres.full 2>&1

Same code, hence same comments as the previous patch.

--D

> +
> +# success, all done
> +status=0
> +exit
> + 
> diff --git a/tests/xfs/528.out b/tests/xfs/528.out
> new file mode 100644
> index 00000000..d0f2c878
> --- /dev/null
> +++ b/tests/xfs/528.out
> @@ -0,0 +1,12 @@
> +QA output created by 528
> +Format and mount fs
> +Inject reduce_max_iextents error tag
> +Create a $srcfile having an extent of length 15 blocks
> +* Write to shared extent
> +Share the extent with $dstfile
> +Buffered write to every other block of $dstfile's shared extent
> +Verify $dstfile's extent count
> +* Funshare shared extent
> +Share the extent with $dstfile
> +Funshare every other block of $dstfile's shared extent
> +Verify $dstfile's extent count
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 627813fe..c85aac6b 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -525,3 +525,4 @@
>  525 auto quick attr
>  526 auto quick dir hardlink symlink
>  527 auto quick
> +528 auto quick reflink
> -- 
> 2.29.2
> 

Re: [PATCH V4 09/11] xfs: Check for extent overflow when remapping an extent

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:20AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when remapping extents from one file's inode fork to another.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/529     | 82 +++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/529.out |  8 +++++
>  tests/xfs/group   |  1 +
>  3 files changed, 91 insertions(+)
>  create mode 100755 tests/xfs/529
>  create mode 100644 tests/xfs/529.out
> 
> diff --git a/tests/xfs/529 b/tests/xfs/529
> new file mode 100755
> index 00000000..f6a5922f
> --- /dev/null
> +++ b/tests/xfs/529
> @@ -0,0 +1,82 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 529
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# remapping extents from one file's inode fork to another.
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/reflink
> +. ./common/inject
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_scratch_reflink
> +_require_xfs_debug
> +_require_xfs_io_command "reflink"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +
> +echo "* Reflink remap extents"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_block_size $SCRATCH_MNT)
> +
> +srcfile=${SCRATCH_MNT}/srcfile
> +dstfile=${SCRATCH_MNT}/dstfile
> +
> +nr_blks=15
> +
> +echo "Create \$srcfile having an extent of length $nr_blks blocks"
> +$XFS_IO_PROG -f -c "pwrite -b $((nr_blks * bsize))  0 $((nr_blks * bsize))" \
> +       -c fsync $srcfile >> $seqres.full
> +
> +echo "Create \$dstfile having an extent of length $nr_blks blocks"
> +$XFS_IO_PROG -f -c "pwrite -b $((nr_blks * bsize))  0 $((nr_blks * bsize))" \
> +       -c fsync $dstfile >> $seqres.full
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Reflink every other block from \$srcfile into \$dstfile"
> +for i in $(seq 1 2 $((nr_blks - 1))); do
> +	$XFS_IO_PROG -f -c "reflink $srcfile $((i * bsize)) $((i * bsize)) $bsize" \
> +	       $dstfile >> $seqres.full 2>&1

_reflink_range?

--D

> +done
> +
> +echo "Verify \$dstfile's extent count"
> +nextents=$($XFS_IO_PROG -c 'stat' $dstfile | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +if (( $nextents > 10 )); then
> +	echo "Extent count overflow check failed: nextents = $nextents"
> +	exit 1
> +fi
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/529.out b/tests/xfs/529.out
> new file mode 100644
> index 00000000..687a8bd2
> --- /dev/null
> +++ b/tests/xfs/529.out
> @@ -0,0 +1,8 @@
> +QA output created by 529
> +* Reflink remap extents
> +Format and mount fs
> +Create $srcfile having an extent of length 15 blocks
> +Create $dstfile having an extent of length 15 blocks
> +Inject reduce_max_iextents error tag
> +Reflink every other block from $srcfile into $dstfile
> +Verify $dstfile's extent count
> diff --git a/tests/xfs/group b/tests/xfs/group
> index c85aac6b..bc3958b3 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -526,3 +526,4 @@
>  526 auto quick dir hardlink symlink
>  527 auto quick
>  528 auto quick reflink
> +529 auto quick reflink
> -- 
> 2.29.2
> 

Re: [PATCH V4 10/11] xfs: Check for extent overflow when swapping extents

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:21AM +0530, Chandan Babu R wrote:
> This test verifies that XFS does not cause inode fork's extent count to
> overflow when swapping forks across two files.
> 
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/530     | 109 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/530.out |  13 ++++++
>  tests/xfs/group   |   1 +
>  3 files changed, 123 insertions(+)
>  create mode 100755 tests/xfs/530
>  create mode 100644 tests/xfs/530.out
> 
> diff --git a/tests/xfs/530 b/tests/xfs/530
> new file mode 100755
> index 00000000..0986d8bf
> --- /dev/null
> +++ b/tests/xfs/530
> @@ -0,0 +1,109 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.

2021? :D

Otherwise this looks ok,
Reviewed-by: Darrick J. Wong <djwong@kernel.org>

--D

> +#
> +# FS QA Test 530
> +#
> +# Verify that XFS does not cause inode fork's extent count to overflow when
> +# swapping forks between files
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_debug
> +_require_xfs_scratch_rmapbt
> +_require_xfs_io_command "fcollapse"
> +_require_xfs_io_command "swapext"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +
> +echo "* Swap extent forks"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_block_size $SCRATCH_MNT)
> +
> +srcfile=${SCRATCH_MNT}/srcfile
> +donorfile=${SCRATCH_MNT}/donorfile
> +
> +echo "Create \$donorfile having an extent of length 67 blocks"
> +$XFS_IO_PROG -f -s -c "pwrite -b $((17 * bsize)) 0 $((17 * bsize))" $donorfile \
> +       >> $seqres.full
> +
> +# After the for loop the donor file will have the following extent layout
> +# | 0-4 | 5 | 6 | 7 | 8 | 9 | 10 |
> +echo "Fragment \$donorfile"
> +for i in $(seq 5 10); do
> +	start_offset=$((i * bsize))
> +	$XFS_IO_PROG -f -c "fcollapse $start_offset $bsize" $donorfile >> $seqres.full
> +done
> +
> +echo "Create \$srcfile having an extent of length 18 blocks"
> +$XFS_IO_PROG -f -s -c "pwrite -b $((18 * bsize)) 0 $((18 * bsize))" $srcfile \
> +       >> $seqres.full
> +
> +echo "Fragment \$srcfile"
> +# After the for loop the src file will have the following extent layout
> +# | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7-10 |
> +for i in $(seq 1 7); do
> +	start_offset=$((i * bsize))
> +	$XFS_IO_PROG -f -c "fcollapse $start_offset $bsize" $srcfile >> $seqres.full
> +done
> +
> +echo "Collect \$donorfile's extent count"
> +donor_nr_exts=$($XFS_IO_PROG -c 'stat' $donorfile | grep nextents)
> +donor_nr_exts=${donor_nr_exts##fsxattr.nextents = }
> +
> +echo "Collect \$srcfile's extent count"
> +src_nr_exts=$($XFS_IO_PROG -c 'stat' $srcfile | grep nextents)
> +src_nr_exts=${src_nr_exts##fsxattr.nextents = }
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Swap \$srcfile's and \$donorfile's extent forks"
> +$XFS_IO_PROG -f -c "swapext $donorfile" $srcfile >> $seqres.full 2>&1
> +
> +echo "Check for \$donorfile's extent count overflow"
> +nextents=$($XFS_IO_PROG -c 'stat' $donorfile | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +
> +if (( $nextents == $src_nr_exts )); then
> +	echo "\$donorfile: Extent count overflow check failed"
> +fi
> +
> +echo "Check for \$srcfile's extent count overflow"
> +nextents=$($XFS_IO_PROG -c 'stat' $srcfile | grep nextents)
> +nextents=${nextents##fsxattr.nextents = }
> +
> +if (( $nextents == $donor_nr_exts )); then
> +	echo "\$srcfile: Extent count overflow check failed"
> +fi
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/530.out b/tests/xfs/530.out
> new file mode 100644
> index 00000000..9f55608b
> --- /dev/null
> +++ b/tests/xfs/530.out
> @@ -0,0 +1,13 @@
> +QA output created by 530
> +* Swap extent forks
> +Format and mount fs
> +Create $donorfile having an extent of length 67 blocks
> +Fragment $donorfile
> +Create $srcfile having an extent of length 18 blocks
> +Fragment $srcfile
> +Collect $donorfile's extent count
> +Collect $srcfile's extent count
> +Inject reduce_max_iextents error tag
> +Swap $srcfile's and $donorfile's extent forks
> +Check for $donorfile's extent count overflow
> +Check for $srcfile's extent count overflow
> diff --git a/tests/xfs/group b/tests/xfs/group
> index bc3958b3..81a15582 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -527,3 +527,4 @@
>  527 auto quick
>  528 auto quick reflink
>  529 auto quick reflink
> +530 auto quick
> -- 
> 2.29.2
> 

Re: [PATCH V4 11/11] xfs: Stress test with bmap_alloc_minlen_extent error tag enabled

[Darrick J. Wong]

On Mon, Jan 18, 2021 at 11:50:22AM +0530, Chandan Babu R wrote:
> This commit adds a stress test that executes fsstress with
> bmap_alloc_minlen_extent error tag enabled.

Haha, yikes.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>

--D


> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> ---
>  tests/xfs/531     | 84 +++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/531.out |  7 ++++
>  tests/xfs/group   |  1 +
>  3 files changed, 92 insertions(+)
>  create mode 100755 tests/xfs/531
>  create mode 100644 tests/xfs/531.out
> 
> diff --git a/tests/xfs/531 b/tests/xfs/531
> new file mode 100755
> index 00000000..fd92c3ea
> --- /dev/null
> +++ b/tests/xfs/531
> @@ -0,0 +1,84 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> +#
> +# FS QA Test 531
> +#
> +# Execute fsstress with bmap_alloc_minlen_extent error tag enabled.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_debug
> +_require_test_program "punch-alternating"
> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> +
> +echo "Format and mount fs"
> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +bsize=$(_get_file_block_size $SCRATCH_MNT)
> +
> +echo "Consume free space"
> +fillerdir=$SCRATCH_MNT/fillerdir
> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> +nr_free_blks=$((nr_free_blks * 90 / 100))
> +
> +_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
> +
> +echo "Create fragmented filesystem"
> +for dentry in $(ls -1 $fillerdir/); do
> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> +done
> +
> +echo "Inject bmap_alloc_minlen_extent error tag"
> +_scratch_inject_error bmap_alloc_minlen_extent 1
> +
> +echo "Scale fsstress args"
> +args=$(_scale_fsstress_args -p $((LOAD_FACTOR * 75)) -n $((TIME_FACTOR * 1000)))
> +
> +echo "Execute fsstress in background"
> +$FSSTRESS_PROG -d $SCRATCH_MNT $args \
> +		 -f bulkstat=0 \
> +		 -f bulkstat1=0 \
> +		 -f fiemap=0 \
> +		 -f getattr=0 \
> +		 -f getdents=0 \
> +		 -f getfattr=0 \
> +		 -f listfattr=0 \
> +		 -f mread=0 \
> +		 -f read=0 \
> +		 -f readlink=0 \
> +		 -f readv=0 \
> +		 -f stat=0 \
> +		 -f aread=0 \
> +		 -f dread=0 > /dev/null 2>&1
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/531.out b/tests/xfs/531.out
> new file mode 100644
> index 00000000..67f40654
> --- /dev/null
> +++ b/tests/xfs/531.out
> @@ -0,0 +1,7 @@
> +QA output created by 531
> +Format and mount fs
> +Consume free space
> +Create fragmented filesystem
> +Inject bmap_alloc_minlen_extent error tag
> +Scale fsstress args
> +Execute fsstress in background
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 81a15582..f4cb5af6 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -528,3 +528,4 @@
>  528 auto quick reflink
>  529 auto quick reflink
>  530 auto quick
> +531 auto stress
> -- 
> 2.29.2
> 

Re: [PATCH V4 03/11] xfs: Check for extent overflow when growing realtime bitmap/summary inodes

[Chandan Babu R]

On 03 Mar 2021 at 23:15, Darrick J. Wong wrote:
> On Mon, Jan 18, 2021 at 11:50:14AM +0530, Chandan Babu R wrote:
>> Verify that XFS does not cause realtime bitmap/summary inode fork's
>> extent count to overflow when growing the realtime volume associated
>> with a filesystem.
>>
>> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
>> ---
>>  tests/xfs/523     | 119 ++++++++++++++++++++++++++++++++++++++++++++++
>>  tests/xfs/523.out |  11 +++++
>>  tests/xfs/group   |   1 +
>>  3 files changed, 131 insertions(+)
>>  create mode 100755 tests/xfs/523
>>  create mode 100644 tests/xfs/523.out
>>
>> diff --git a/tests/xfs/523 b/tests/xfs/523
>> new file mode 100755
>> index 00000000..145f0ff6
>> --- /dev/null
>> +++ b/tests/xfs/523
>> @@ -0,0 +1,119 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
>> +#
>> +# FS QA Test 523
>> +#
>> +# Verify that XFS does not cause bitmap/summary inode fork's extent count to
>> +# overflow when growing an the realtime volume of the filesystem.
>> +#
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1	# failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> +	cd /
>> +	_scratch_unmount >> $seqres.full 2>&1
>> +	test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1
>> +	rm -f $tmp.* $TEST_DIR/$seq.rtvol
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/inject
>> +. ./common/populate
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs xfs
>> +_require_test
>> +_require_xfs_debug
>> +_require_test_program "punch-alternating"
>> +_require_xfs_io_error_injection "reduce_max_iextents"
>> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
>> +_require_scratch_nocheck
>
> Might want a comment about why this test doesn't _require_realtime.

Sure. I will do that.

>
> Hm.  What /does/ happen if the kernel xfs driver wasn't built with
> realtime support?  I'll investigate and report back.

_scratch_mount invocation later in the script fails and hence the script exits
with a failure status.

>
>> +
>> +echo "* Test extending rt inodes"
>> +
>> +_scratch_mkfs | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
>> +. $tmp.mkfs
>> +
>> +echo "Create fake rt volume"
>> +nr_bitmap_blks=25
>> +nr_bits=$((nr_bitmap_blks * dbsize * 8))
>> +rtextsz=$dbsize
>> +rtdevsz=$((nr_bits * rtextsz))
>> +truncate -s $rtdevsz $TEST_DIR/$seq.rtvol
>> +rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol)
>> +
>> +echo "Format and mount rt volume"
>> +
>> +export USE_EXTERNAL=yes
>> +export SCRATCH_RTDEV=$rtdev
>> +_scratch_mkfs -d size=$((1024 * 1024 * 1024)) \
>> +	      -r size=2M,extsize=${rtextsz} >> $seqres.full
>> +_scratch_mount >> $seqres.full
>> +
>> +echo "Consume free space"
>> +fillerdir=$SCRATCH_MNT/fillerdir
>> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
>> +nr_free_blks=$((nr_free_blks * 90 / 100))
>> +
>> +_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
>> +
>> +echo "Create fragmented filesystem"
>> +for dentry in $(ls -1 $fillerdir/); do
>> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
>> +done
>> +
>> +echo "Inject reduce_max_iextents error tag"
>> +_scratch_inject_error reduce_max_iextents 1
>> +
>> +echo "Inject bmap_alloc_minlen_extent error tag"
>> +_scratch_inject_error bmap_alloc_minlen_extent 1
>> +
>> +echo "Grow realtime volume"
>> +$XFS_GROWFS_PROG -r $SCRATCH_MNT >> $seqres.full 2>&1
>> +if [[ $? == 0 ]]; then
>> +	echo "Growfs succeeded; should have failed."
>> +	exit 1
>> +fi
>> +
>> +_scratch_unmount >> $seqres.full
>> +
>> +echo "Verify rbmino's and rsumino's extent count"
>> +for rtino in rbmino rsumino; do
>> +	ino=$(_scratch_xfs_db -c sb -c "print $rtino")
>> +	ino=${ino##${rtino} = }
>> +	echo "$rtino = $ino" >> $seqres.full
>> +
>> +	nextents=$(_scratch_get_iext_count $ino data || \
>> +			_fail "Unable to obtain inode fork's extent count")
>> +	if (( $nextents > 10 )); then
>> +		echo "Extent count overflow check failed: nextents = $nextents"
>> +		exit 1
>> +	fi
>> +done
>> +
>> +echo "Check filesystem"
>> +_check_xfs_filesystem $SCRATCH_DEV none $rtdev
>> +
>> +losetup -d $rtdev
>> +rm -f $TEST_DIR/$seq.rtvol
>> +
>> +export USE_EXTERNAL=""
>> +export SCRATCH_RTDEV=""
>
> No need to clear these, but with those two nits fixed, this looks all right.
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>

Thanks for the review!

I also noticed a bug in the script when it is executed on a 1k block sized
filesystem. The second invocation of _scratch_mkfs ends up creating a 4k
blocksized filesystem since the block size was not explicitly passed as an
argument. Also, if the block size is less than 4k, we have to set realtime
block size to 4k.

I have fixed these bugs and will include the fixes in the next version
of the patchset.

>
> --D
>
>> +
>> +# success, all done
>> +status=0
>> +exit
>> diff --git a/tests/xfs/523.out b/tests/xfs/523.out
>> new file mode 100644
>> index 00000000..7df02970
>> --- /dev/null
>> +++ b/tests/xfs/523.out
>> @@ -0,0 +1,11 @@
>> +QA output created by 523
>> +* Test extending rt inodes
>> +Create fake rt volume
>> +Format and mount rt volume
>> +Consume free space
>> +Create fragmented filesystem
>> +Inject reduce_max_iextents error tag
>> +Inject bmap_alloc_minlen_extent error tag
>> +Grow realtime volume
>> +Verify rbmino's and rsumino's extent count
>> +Check filesystem
>> diff --git a/tests/xfs/group b/tests/xfs/group
>> index 1831f0b5..018c70ef 100644
>> --- a/tests/xfs/group
>> +++ b/tests/xfs/group
>> @@ -520,3 +520,4 @@
>>  520 auto quick reflink
>>  521 auto quick realtime growfs
>>  522 auto quick quota
>> +523 auto quick realtime growfs
>> --
>> 2.29.2
>>


--
chandan

Re: [PATCH V4 06/11] xfs: Check for extent overflow when adding/removing dir entries

[Chandan Babu R]

On 03 Mar 2021 at 23:31, Darrick J. Wong wrote:
> On Mon, Jan 18, 2021 at 11:50:17AM +0530, Chandan Babu R wrote:
>> This test verifies that XFS does not cause inode fork's extent count to
>> overflow when adding/removing directory entries.
>>
>> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
>> ---
>>  tests/xfs/526     | 186 ++++++++++++++++++++++++++++++++++++++++++++++
>>  tests/xfs/526.out |  17 +++++
>>  tests/xfs/group   |   1 +
>>  3 files changed, 204 insertions(+)
>>  create mode 100755 tests/xfs/526
>>  create mode 100644 tests/xfs/526.out
>>
>> diff --git a/tests/xfs/526 b/tests/xfs/526
>> new file mode 100755
>> index 00000000..5a789d61
>> --- /dev/null
>> +++ b/tests/xfs/526
>> @@ -0,0 +1,186 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
>> +#
>> +# FS QA Test 526
>> +#
>> +# Verify that XFS does not cause inode fork's extent count to overflow when
>> +# adding/removing directory entries.
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1	# failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> +	cd /
>> +	rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/inject
>> +. ./common/populate
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs xfs
>> +_require_scratch
>> +_require_xfs_debug
>> +_require_test_program "punch-alternating"
>> +_require_xfs_io_error_injection "reduce_max_iextents"
>> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
>> +
>> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
>> +. $tmp.mkfs
>> +
>> +# Filesystems with directory block size greater than one FSB will not be tested,
>> +# since "7 (i.e. XFS_DA_NODE_MAXDEPTH + 1 data block + 1 free block) * 2 (fsb
>> +# count) = 14" is greater than the pseudo max extent count limit of 10.
>> +# Extending the pseudo max limit won't help either.  Consider the case where 1
>> +# FSB is 1k in size and 1 dir block is 64k in size (i.e. fsb count = 64). In
>> +# this case, the pseudo max limit has to be greater than 7 * 64 = 448 extents.
>> +if (( $dbsize != $dirbsize )); then
>> +	_notrun "FSB size ($dbsize) and directory block size ($dirbsize) do not match"
>> +fi
>
> But what about the case where fsb is 1k and dirblocks are 4k? :)
>
> I admit I'm reacting to my expectation that we would _notrun here based
> on the output of a more computation, not just fsb != dirblocksize.  But
> I dunno, maybe you've already worked that out?

Ok. I will replace the above with the following,

if (( $dirbsize > $dbsize )); then
        _notrun "Directory block size ($dirbsize) is larger than FSB size ($dbsize)"
fi

>
> (The rest of the test looks good to me.)
>
> --D

--
chandan

Re: [PATCH V4 03/11] xfs: Check for extent overflow when growing realtime bitmap/summary inodes

[Darrick J. Wong]

On Thu, Mar 04, 2021 at 05:28:30PM +0530, Chandan Babu R wrote:
> On 03 Mar 2021 at 23:15, Darrick J. Wong wrote:
> > On Mon, Jan 18, 2021 at 11:50:14AM +0530, Chandan Babu R wrote:
> >> Verify that XFS does not cause realtime bitmap/summary inode fork's
> >> extent count to overflow when growing the realtime volume associated
> >> with a filesystem.
> >>
> >> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> >> ---
> >>  tests/xfs/523     | 119 ++++++++++++++++++++++++++++++++++++++++++++++
> >>  tests/xfs/523.out |  11 +++++
> >>  tests/xfs/group   |   1 +
> >>  3 files changed, 131 insertions(+)
> >>  create mode 100755 tests/xfs/523
> >>  create mode 100644 tests/xfs/523.out
> >>
> >> diff --git a/tests/xfs/523 b/tests/xfs/523
> >> new file mode 100755
> >> index 00000000..145f0ff6
> >> --- /dev/null
> >> +++ b/tests/xfs/523
> >> @@ -0,0 +1,119 @@
> >> +#! /bin/bash
> >> +# SPDX-License-Identifier: GPL-2.0
> >> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
> >> +#
> >> +# FS QA Test 523
> >> +#
> >> +# Verify that XFS does not cause bitmap/summary inode fork's extent count to
> >> +# overflow when growing an the realtime volume of the filesystem.
> >> +#
> >> +seq=`basename $0`
> >> +seqres=$RESULT_DIR/$seq
> >> +echo "QA output created by $seq"
> >> +
> >> +here=`pwd`
> >> +tmp=/tmp/$$
> >> +status=1	# failure is the default!
> >> +trap "_cleanup; exit \$status" 0 1 2 3 15
> >> +
> >> +_cleanup()
> >> +{
> >> +	cd /
> >> +	_scratch_unmount >> $seqres.full 2>&1
> >> +	test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1
> >> +	rm -f $tmp.* $TEST_DIR/$seq.rtvol
> >> +}
> >> +
> >> +# get standard environment, filters and checks
> >> +. ./common/rc
> >> +. ./common/filter
> >> +. ./common/inject
> >> +. ./common/populate
> >> +
> >> +# remove previous $seqres.full before test
> >> +rm -f $seqres.full
> >> +
> >> +# real QA test starts here
> >> +
> >> +_supported_fs xfs
> >> +_require_test
> >> +_require_xfs_debug
> >> +_require_test_program "punch-alternating"
> >> +_require_xfs_io_error_injection "reduce_max_iextents"
> >> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> >> +_require_scratch_nocheck
> >
> > Might want a comment about why this test doesn't _require_realtime.
> 
> Sure. I will do that.
> 
> >
> > Hm.  What /does/ happen if the kernel xfs driver wasn't built with
> > realtime support?  I'll investigate and report back.
> 
> _scratch_mount invocation later in the script fails and hence the script exits
> with a failure status.

That kind of regression will cause bug reports.  Can you make it so that
fstests skips the test if the rt mount fails?  e.g.

_try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"

--D

> 
> >
> >> +
> >> +echo "* Test extending rt inodes"
> >> +
> >> +_scratch_mkfs | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
> >> +. $tmp.mkfs
> >> +
> >> +echo "Create fake rt volume"
> >> +nr_bitmap_blks=25
> >> +nr_bits=$((nr_bitmap_blks * dbsize * 8))
> >> +rtextsz=$dbsize
> >> +rtdevsz=$((nr_bits * rtextsz))
> >> +truncate -s $rtdevsz $TEST_DIR/$seq.rtvol
> >> +rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol)
> >> +
> >> +echo "Format and mount rt volume"
> >> +
> >> +export USE_EXTERNAL=yes
> >> +export SCRATCH_RTDEV=$rtdev
> >> +_scratch_mkfs -d size=$((1024 * 1024 * 1024)) \
> >> +	      -r size=2M,extsize=${rtextsz} >> $seqres.full
> >> +_scratch_mount >> $seqres.full
> >> +
> >> +echo "Consume free space"
> >> +fillerdir=$SCRATCH_MNT/fillerdir
> >> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> >> +nr_free_blks=$((nr_free_blks * 90 / 100))
> >> +
> >> +_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
> >> +
> >> +echo "Create fragmented filesystem"
> >> +for dentry in $(ls -1 $fillerdir/); do
> >> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> >> +done
> >> +
> >> +echo "Inject reduce_max_iextents error tag"
> >> +_scratch_inject_error reduce_max_iextents 1
> >> +
> >> +echo "Inject bmap_alloc_minlen_extent error tag"
> >> +_scratch_inject_error bmap_alloc_minlen_extent 1
> >> +
> >> +echo "Grow realtime volume"
> >> +$XFS_GROWFS_PROG -r $SCRATCH_MNT >> $seqres.full 2>&1
> >> +if [[ $? == 0 ]]; then
> >> +	echo "Growfs succeeded; should have failed."
> >> +	exit 1
> >> +fi
> >> +
> >> +_scratch_unmount >> $seqres.full
> >> +
> >> +echo "Verify rbmino's and rsumino's extent count"
> >> +for rtino in rbmino rsumino; do
> >> +	ino=$(_scratch_xfs_db -c sb -c "print $rtino")
> >> +	ino=${ino##${rtino} = }
> >> +	echo "$rtino = $ino" >> $seqres.full
> >> +
> >> +	nextents=$(_scratch_get_iext_count $ino data || \
> >> +			_fail "Unable to obtain inode fork's extent count")
> >> +	if (( $nextents > 10 )); then
> >> +		echo "Extent count overflow check failed: nextents = $nextents"
> >> +		exit 1
> >> +	fi
> >> +done
> >> +
> >> +echo "Check filesystem"
> >> +_check_xfs_filesystem $SCRATCH_DEV none $rtdev
> >> +
> >> +losetup -d $rtdev
> >> +rm -f $TEST_DIR/$seq.rtvol
> >> +
> >> +export USE_EXTERNAL=""
> >> +export SCRATCH_RTDEV=""
> >
> > No need to clear these, but with those two nits fixed, this looks all right.
> > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> 
> Thanks for the review!
> 
> I also noticed a bug in the script when it is executed on a 1k block sized
> filesystem. The second invocation of _scratch_mkfs ends up creating a 4k
> blocksized filesystem since the block size was not explicitly passed as an
> argument. Also, if the block size is less than 4k, we have to set realtime
> block size to 4k.
> 
> I have fixed these bugs and will include the fixes in the next version
> of the patchset.
> 
> >
> > --D
> >
> >> +
> >> +# success, all done
> >> +status=0
> >> +exit
> >> diff --git a/tests/xfs/523.out b/tests/xfs/523.out
> >> new file mode 100644
> >> index 00000000..7df02970
> >> --- /dev/null
> >> +++ b/tests/xfs/523.out
> >> @@ -0,0 +1,11 @@
> >> +QA output created by 523
> >> +* Test extending rt inodes
> >> +Create fake rt volume
> >> +Format and mount rt volume
> >> +Consume free space
> >> +Create fragmented filesystem
> >> +Inject reduce_max_iextents error tag
> >> +Inject bmap_alloc_minlen_extent error tag
> >> +Grow realtime volume
> >> +Verify rbmino's and rsumino's extent count
> >> +Check filesystem
> >> diff --git a/tests/xfs/group b/tests/xfs/group
> >> index 1831f0b5..018c70ef 100644
> >> --- a/tests/xfs/group
> >> +++ b/tests/xfs/group
> >> @@ -520,3 +520,4 @@
> >>  520 auto quick reflink
> >>  521 auto quick realtime growfs
> >>  522 auto quick quota
> >> +523 auto quick realtime growfs
> >> --
> >> 2.29.2
> >>
> 
> 
> --
> chandan

Re: [PATCH V4 01/11] common/xfs: Add a helper to get an inode fork's extent count

[Chandan Babu R]

On 03 Mar 2021 at 23:00, Darrick J. Wong wrote:
> On Mon, Jan 18, 2021 at 11:50:12AM +0530, Chandan Babu R wrote:
>> This commit adds the helper _scratch_get_iext_count() which returns an
>> inode fork's extent count.
>>
>> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
>> ---
>>  common/xfs | 22 ++++++++++++++++++++++
>>  1 file changed, 22 insertions(+)
>>
>> diff --git a/common/xfs b/common/xfs
>> index 3f5c14ba..641d6195 100644
>> --- a/common/xfs
>> +++ b/common/xfs
>> @@ -903,6 +903,28 @@ _scratch_get_bmx_prefix() {
>>  	return 1
>>  }
>>
>> +_scratch_get_iext_count()
>> +{
>> +	ino=$1
>> +	whichfork=$2
>
> Function variables should be declared with 'local' so they don't bleed
> into the global namespace (yay bash!), e.g.
>
> 	local ino="$1"

Sorry, I forgot about this. I will fix this up.

>
> Also, now that Eric has landed the xfs_db 'path' command upstream, you
> might consider using it:
>
> 	_scratch_xfs_get_metadata_field "core.nextents" "path /windows/system.ini"
>

In this patchset _scratch_get_iext_count() is being used to get extent counts
of anonymous inodes i.e. inodes which do not have an entry in the filesytem
namespace (e.g. Quota and RT bitmap/summary inodes). Hence the 'path' command
won't be useful in this case.

>> +
>> +	case $whichfork in
>> +		"attr")
>> +			field=core.naextents
>> +			;;
>> +		"data")
>> +			field=core.nextents
>> +			;;
>> +		*)
>> +			return 1
>> +	esac
>> +
>> +	nextents=$(_scratch_xfs_db  -c "inode $ino" -c "print $field")
>> +	nextents=${nextents##${field} = }
>
> _scratch_xfs_get_metadata_field?

Sure, I will make use of the above mentioned helper.

--
chandan

Re: [PATCH V4 02/11] xfs: Check for extent overflow when trivally adding a new extent

[Chandan Babu R]

On 03 Mar 2021 at 23:08, Darrick J. Wong wrote:
> On Mon, Jan 18, 2021 at 11:50:13AM +0530, Chandan Babu R wrote:
>> This test verifies that XFS does not cause inode fork's extent count to
>> overflow when adding a single extent while there's no possibility of
>> splitting an existing mapping.
>>
>> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
>> ---
>>  tests/xfs/522     | 173 ++++++++++++++++++++++++++++++++++++++++++++++
>>  tests/xfs/522.out |  20 ++++++
>>  tests/xfs/group   |   1 +
>>  3 files changed, 194 insertions(+)
>>  create mode 100755 tests/xfs/522
>>  create mode 100644 tests/xfs/522.out
>>
>> diff --git a/tests/xfs/522 b/tests/xfs/522
>> new file mode 100755
>> index 00000000..33f0591e
>> --- /dev/null
>> +++ b/tests/xfs/522
>> @@ -0,0 +1,173 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
>> +#
>> +# FS QA Test 522
>> +#
>> +# Verify that XFS does not cause inode fork's extent count to overflow when
>> +# adding a single extent while there's no possibility of splitting an existing
>> +# mapping.
>> +
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1	# failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> +	cd /
>> +	rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/quota
>> +. ./common/inject
>> +. ./common/populate
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs xfs
>> +_require_scratch
>> +_require_xfs_quota
>> +_require_xfs_debug
>> +_require_test_program "punch-alternating"
>> +_require_xfs_io_command "falloc"
>> +_require_xfs_io_error_injection "reduce_max_iextents"
>> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
>> +
>> +echo "Format and mount fs"
>> +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full
>> +_scratch_mount -o uquota >> $seqres.full
>> +
>> +bsize=$(_get_file_block_size $SCRATCH_MNT)
>> +
>> +echo "* Delalloc to written extent conversion"
>
> What happens if delalloc is disabled?  e.g. the mkfs parameters have
> extent size hints set on the root directory?
>
> (I think it'll be fine since you write every other block...)

Yes, XFS ends up doing Direct IO and the file blocks are still fragmented
causing extent count to increase beyond the pseudo max limit of 10.

>
>> +testfile=$SCRATCH_MNT/testfile
>> +
>> +echo "Inject reduce_max_iextents error tag"
>> +_scratch_inject_error reduce_max_iextents 1
>> +
>> +nr_blks=$((15 * 2))
>> +
>> +echo "Create fragmented file"
>> +for i in $(seq 0 2 $((nr_blks - 1))); do
>> +	$XFS_IO_PROG -f -s -c "pwrite $((i * bsize)) $bsize" $testfile \
>> +	       >> $seqres.full 2>&1
>> +	[[ $? != 0 ]] && break
>> +done
>> +
>> +echo "Verify \$testfile's extent count"
>> +
>> +nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
>> +nextents=${nextents##fsxattr.nextents = }
>
> /me wonders if it's time for a helper to extract these fields...

Thanks for the suggestion. I will include a helper function in the next
version of the patchset.

>
>> +if (( $nextents > 10 )); then
>> +	echo "Extent count overflow check failed: nextents = $nextents"
>> +	exit 1
>> +fi
>> +
>> +rm $testfile
>> +
>> +echo "* Fallocate unwritten extents"
>> +
>> +echo "Fallocate fragmented file"
>> +for i in $(seq 0 2 $((nr_blks - 1))); do
>> +	$XFS_IO_PROG -f -c "falloc $((i * bsize)) $bsize" $testfile \
>> +	       >> $seqres.full 2>&1
>> +	[[ $? != 0 ]] && break
>> +done
>> +
>> +echo "Verify \$testfile's extent count"
>> +
>> +nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
>> +nextents=${nextents##fsxattr.nextents = }
>> +if (( $nextents > 10 )); then
>> +	echo "Extent count overflow check failed: nextents = $nextents"
>> +	exit 1
>> +fi
>> +
>> +rm $testfile
>> +
>> +echo "* Directio write"
>> +
>> +echo "Create fragmented file via directio writes"
>> +for i in $(seq 0 2 $((nr_blks - 1))); do
>> +	$XFS_IO_PROG -d -s -f -c "pwrite $((i * bsize)) $bsize" $testfile \
>> +	       >> $seqres.full 2>&1
>> +	[[ $? != 0 ]] && break
>> +done
>> +
>> +echo "Verify \$testfile's extent count"
>> +
>> +nextents=$($XFS_IO_PROG -f -c 'stat' $testfile | grep -i nextents)
>> +nextents=${nextents##fsxattr.nextents = }
>> +if (( $nextents > 10 )); then
>> +	echo "Extent count overflow check failed: nextents = $nextents"
>> +	exit 1
>> +fi
>> +
>> +rm $testfile
>> +
>> +echo "* Extend quota inodes"
>> +
>> +echo "Disable reduce_max_iextents error tag"
>> +_scratch_inject_error reduce_max_iextents 0
>> +
>> +echo "Consume free space"
>> +fillerdir=$SCRATCH_MNT/fillerdir
>> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
>> +nr_free_blks=$((nr_free_blks * 90 / 100))
>> +
>> +_fill_fs $((bsize * nr_free_blks)) $fillerdir $bsize 0 >> $seqres.full 2>&1
>> +
>> +echo "Create fragmented filesystem"
>> +for dentry in $(ls -1 $fillerdir/); do
>> +	$here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
>> +done
>> +
>> +echo "Inject reduce_max_iextents error tag"
>> +_scratch_inject_error reduce_max_iextents 1
>> +
>> +echo "Inject bmap_alloc_minlen_extent error tag"
>> +_scratch_inject_error bmap_alloc_minlen_extent 1
>
> Are we testing to see if the fs blows up when the quota inodes hit max
> iext count?  If so, please put that in the comments for this section.

Sure, I will add a description.

>
>> +
>> +nr_blks=20
>> +
>> +# This is a rough calculation; It doesn't take block headers into
>> +# consideration.
>> +# gdb -batch vmlinux -ex 'print sizeof(struct xfs_dqblk)'
>> +# $1 = 136
>> +nr_quotas_per_block=$((bsize / 136))
>> +nr_quotas=$((nr_quotas_per_block * nr_blks))
>> +
>> +echo "Extend uquota file"
>> +for i in $(seq 0 $nr_quotas_per_block $nr_quotas); do
>> +	chown $i $testfile >> $seqres.full 2>&1
>> +	[[ $? != 0 ]] && break
>> +done
>> +
>> +_scratch_unmount >> $seqres.full
>> +
>> +echo "Verify uquota inode's extent count"
>> +uquotino=$(_scratch_xfs_db -c sb -c "print uquotino")
>> +uquotino=${uquotino##uquotino = }
>
> _scratch_xfs_get_metadata_field

I will fix this.

--
chandan

Re: [PATCH V4 07/11] xfs: Check for extent overflow when writing to unwritten extent

[Chandan Babu R]

On 03 Mar 2021 at 23:35, Darrick J. Wong wrote:
> On Mon, Jan 18, 2021 at 11:50:18AM +0530, Chandan Babu R wrote:
>> This test verifies that XFS does not cause inode fork's extent count to
>> overflow when writing to an unwritten extent.
>>
>> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
>> ---
>>  tests/xfs/527     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
>>  tests/xfs/527.out | 11 ++++++
>>  tests/xfs/group   |  1 +
>>  3 files changed, 101 insertions(+)
>>  create mode 100755 tests/xfs/527
>>  create mode 100644 tests/xfs/527.out
>>
>> diff --git a/tests/xfs/527 b/tests/xfs/527
>> new file mode 100755
>> index 00000000..cd67bce4
>> --- /dev/null
>> +++ b/tests/xfs/527
>> @@ -0,0 +1,89 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2020 Chandan Babu R.  All Rights Reserved.
>> +#
>> +# FS QA Test 527
>> +#
>> +# Verify that XFS does not cause inode fork's extent count to overflow when
>> +# writing to an unwritten extent.
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1	# failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> +	cd /
>> +	rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/inject
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs xfs
>> +_require_scratch
>> +_require_xfs_debug
>> +_require_xfs_io_command "falloc"
>> +_require_xfs_io_error_injection "reduce_max_iextents"
>> +
>> +echo "Format and mount fs"
>> +_scratch_mkfs_sized $((1024 * 1024 * 1024)) >> $seqres.full
>> +_scratch_mount >> $seqres.full
>> +
>> +bsize=$(_get_file_block_size $SCRATCH_MNT)
>> +
>> +testfile=${SCRATCH_MNT}/testfile
>> +
>> +echo "Inject reduce_max_iextents error tag"
>> +_scratch_inject_error reduce_max_iextents 1
>> +
>> +nr_blks=15
>> +
>> +for io in Buffered Direct; do
>
> # First test buffered writes, then direct writes.
> for $xfs_io_flag in "" "-d"; do ?

The keywords "Buffered" and "Direct" are also being used in the log
messages printed ...

>
>> +	echo "* $io write to unwritten extent"

... For example, In the above line, $io would be replaced by one of "Buffered"
and "Direct" ...

>> +
>> +	echo "Fallocate $nr_blks blocks"
>> +	$XFS_IO_PROG -f -c "falloc 0 $((nr_blks * bsize))" $testfile >> $seqres.full
>> +
>> +	if [[ $io == "Buffered" ]]; then
>> +		xfs_io_flag=""
>> +	else
>> +		xfs_io_flag="-d"
>> +	fi
>
> ...because then you can skip this part.
>
>> +
>> +	echo "$io write to every other block of fallocated space"

... Same logic is used above.

>> +	for i in $(seq 1 2 $((nr_blks - 1))); do
>> +		$XFS_IO_PROG -f -s $xfs_io_flag -c "pwrite $((i * bsize)) $bsize" \
>> +		       $testfile >> $seqres.full 2>&1
>> +		[[ $? != 0 ]] && break
>> +	done
>> +
>> +	echo "Verify \$testfile's extent count"
>> +	nextents=$($XFS_IO_PROG -c 'stat' $testfile | grep nextents)
>> +	nextents=${nextents##fsxattr.nextents = }
>> +	if (( $nextents > 10 )); then
>> +		echo "Extent count overflow check failed: nextents = $nextents"
>> +		exit 1
>> +	fi
>> +
>> +	rm $testfile
>> +done
>> +
>> +# super_block->s_wb_err will have a newer seq value when compared to "/"'s
>> +# file->f_sb_err. Consume it here so that xfs_scrub can does not error out.
>> +$XFS_IO_PROG -c syncfs $SCRATCH_MNT >> $seqres.full 2>&1
>
> I wonder, should _check_xfs_filesystem should syncfs to clear old EIOs
> before running xfs_scrub?  I occasionally see this pop up on
> generic/204.

I agree with the idea of including a call to syncfs in
_check_xfs_filesystem. An error message can be logged if syncfs returns a
failure status. This will also prevent xfs_scrub from aborting and help in
providing information about any corrupt on-disk data structures.

I will include this in the next version of the patchset.

--
chandan