linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes
@ 2021-09-01  0:11 Darrick J. Wong
  2021-09-01  0:11 ` [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file Darrick J. Wong
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-01  0:11 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

Hi all,

Add regression tests to trigger some bugs in the realtime allocator that
were fixed in kernel 5.13 and 5.14.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-fixes
---
 tests/xfs/774     |   80 ++++++++++++++++++++++++++
 tests/xfs/774.out |    5 ++
 tests/xfs/775     |  165 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/775.out |    3 +
 tests/xfs/776     |   57 ++++++++++++++++++
 tests/xfs/776.out |    5 ++
 tests/xfs/779     |  112 ++++++++++++++++++++++++++++++++++++
 tests/xfs/779.out |    2 +
 8 files changed, 429 insertions(+)
 create mode 100755 tests/xfs/774
 create mode 100644 tests/xfs/774.out
 create mode 100755 tests/xfs/775
 create mode 100644 tests/xfs/775.out
 create mode 100755 tests/xfs/776
 create mode 100644 tests/xfs/776.out
 create mode 100755 tests/xfs/779
 create mode 100644 tests/xfs/779.out


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file
  2021-09-01  0:11 [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes Darrick J. Wong
@ 2021-09-01  0:11 ` Darrick J. Wong
  2021-09-05 14:28   ` Eryu Guan
  2021-09-01  0:11 ` [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs Darrick J. Wong
  2021-09-01  0:11 ` [PATCH 3/3] xfs: test adding realtime sections to filesystem Darrick J. Wong
  2 siblings, 1 reply; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-01  0:11 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test for the two realtime allocator bug fixes:

xfs: adjust rt allocation minlen when extszhint > rtextsize
xfs: retry allocations when locality-based search fails

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/775     |  165 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/775.out |    3 +
 2 files changed, 168 insertions(+)
 create mode 100755 tests/xfs/775
 create mode 100644 tests/xfs/775.out


diff --git a/tests/xfs/775 b/tests/xfs/775
new file mode 100755
index 00000000..e8e6b728
--- /dev/null
+++ b/tests/xfs/775
@@ -0,0 +1,165 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021, Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 775
+#
+# Regression test for commits:
+#
+# 9d5e8492eee0 ("xfs: adjust rt allocation minlen when extszhint > rtextsize")
+# 676a659b60af ("xfs: retry allocations when locality-based search fails")
+#
+# The first bug occurs when an extent size hint is set on a realtime file.
+# xfs_bmapi_rtalloc adjusts the offset and length of the allocation request to
+# try to satisfy the hint, but doesn't adjust minlen to match.  If the
+# allocator finds free space that isn't large enough to map even a single block
+# of the original request, bmapi_write will return ENOSPC and the write fails
+# even though there's plenty of space.
+#
+# The second bug occurs when an extent size hint is set on a file, we ask to
+# allocate blocks in an empty region immediately adjacent to a previous
+# allocation, and the nearest available free space isn't anywhere near the
+# previous allocation, the near allocator will give up and return ENOSPC, even
+# if there's sufficient free realtime extents to satisfy the allocation
+# request.
+#
+# Both bugs can be exploited by the same user call sequence, so here's a
+# targeted test that runs in less time than the reproducers that are listed in
+# the fix patches themselves.
+#
+. ./common/preamble
+_begin_fstest auto quick rw realtime
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+}
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_require_scratch
+_require_realtime
+_require_xfs_io_command "falloc"
+_require_xfs_io_command "fpunch"
+_require_test_program "punch-alternating"
+
+fill_rtdev()
+{
+	file=$1
+
+	filesize=`_get_available_space $SCRATCH_MNT`
+	$XFS_IO_PROG -f -c "truncate $filesize" -c "falloc 0 $filesize" $file
+
+	chunks=20
+	chunksizemb=$((filesize / chunks / 1048576))
+	seq 1 $chunks | while read f; do
+		echo "$((f * chunksizemb)) file size $f / 20"
+		$XFS_IO_PROG -fc "falloc -k $(( (f - 1) * chunksizemb))m ${chunksizemb}m" $file
+	done
+
+	chunks=100
+	chunksizemb=$((filesize / chunks / 1048576))
+	seq 80 $chunks | while read f; do
+		echo "$((f * chunksizemb)) file size $f / $chunks"
+		$XFS_IO_PROG -fc "falloc -k $(( (f - 1) * chunksizemb))m ${chunksizemb}m" $file
+	done
+
+	filesizemb=$((filesize / 1048576))
+	$XFS_IO_PROG -fc "falloc -k 0 ${filesizemb}m" $file
+
+	# Try again anyway
+	avail=`_get_available_space $SCRATCH_MNT`
+	$XFS_IO_PROG -fc "pwrite -S 0x65 0 $avail" ${file}
+}
+
+echo "Format and mount"
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+
+# This is a test of the rt allocator; force all files to be created realtime
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+# Set the extent size hint larger than the realtime extent size.  This is
+# necessary to exercise the minlen constraints on the realtime allocator.
+fsbsize=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep geom.bsize | awk '{print $3}')
+rtextsize_blks=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep geom.rtextsize | awk '{print $3}')
+extsize=$((2 * rtextsize_blks * fsbsize))
+
+echo "rtextsize_blks=$rtextsize_blks extsize=$extsize" >> $seqres.full
+_xfs_force_bdev realtime $SCRATCH_MNT
+$XFS_IO_PROG -c "extsize $extsize" $SCRATCH_MNT
+
+# Compute the geometry of the test files we're going to create.  Realtime
+# volumes are simple, which means that we can control the space allocations
+# exactly to exploit bugs!
+#
+# Since this is a test of the near rt allocator, we need to set up the test to
+# have a victim file with at least one rt extent allocated to it and enough
+# free space to allocate at least one more rt extent at an adjacent file
+# offset.  The free space must not be immediately adjacent to the the first
+# extent that we allocate to the victim file, and it must not be large enough
+# to satisfy the entire allocation request all at once.
+#
+# Our free space fragmentation strategy is the usual fallocate-and-punch swiss
+# cheese file, which means the free space is split into five sections:
+#
+# The first will be remapped into the victim file.
+#
+# The second section exists to prevent the free extents from being adjacent to
+# the first section.  It will be very large, since we allocate all the rt
+# space.
+#
+# The last three sections will have every other rt extent punched out to create
+# some free space.
+remap_sz=$((extsize * 2))
+required_sz=$((5 * remap_sz))
+free_rtspace=$(_get_available_space $SCRATCH_MNT)
+if [ $free_rtspace -lt $required_sz ]; then
+	_notrun "Insufficient free space on rt volume.  Needed $required_sz, saw $free_rtspace."
+fi
+
+# Allocate all the space on the rt volume so that we can control space
+# allocations exactly.
+fill_rtdev $SCRATCH_MNT/bigfile &>> $seqres.full
+
+# We need at least 4 remap sections to proceed
+bigfile_sz=$(stat -c '%s' $SCRATCH_MNT/bigfile)
+if [ $bigfile_sz -lt $required_sz ]; then
+	_notrun "Free space control file needed $required_sz bytes, got $bigfile_sz."
+fi
+
+# Remap the first remap section to a victim file.
+$XFS_IO_PROG -c "fpunch 0 $remap_sz" $SCRATCH_MNT/bigfile
+$XFS_IO_PROG -f -c "truncate $required_sz" -c "falloc 0 $remap_sz" $SCRATCH_MNT/victim
+
+# Punch out every other extent of the last two sections, to fragment free space.
+frag_sz=$((remap_sz * 3))
+punch_off=$((bigfile_sz - frag_sz))
+$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / fsbsize)) -i $((rtextsize_blks * 2)) -s $rtextsize_blks
+
+# Make sure we have some free rtextents.
+free_rtx=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep counts.freertx | awk '{print $3}')
+if [ $free_rtx -eq 0 ]; then
+	echo "Expected fragmented free rt space, found none."
+fi
+
+# Try to double the amount of blocks in the victim file.  On a buggy kernel,
+# the rt allocator will fail immediately with ENOSPC even though we left enough
+# free space for the write will complete fully.
+echo "Try to write a bunch of stuff to the fragmented rt space"
+$XFS_IO_PROG -c "pwrite -S 0x63 -b $remap_sz $remap_sz $remap_sz" -c stat $SCRATCH_MNT/victim >> $seqres.full
+
+# The victim file should own at least two sections' worth of blocks.
+victim_sectors=$(stat -c '%b' $SCRATCH_MNT/victim)
+victim_space_usage=$((victim_sectors * 512))
+expected_usage=$((remap_sz * 2))
+
+if [ $victim_space_usage -lt $expected_usage ]; then
+	echo "Victim file should be using at least $expected_usage bytes, saw $victim_space_usage."
+fi
+
+status=0
+exit
diff --git a/tests/xfs/775.out b/tests/xfs/775.out
new file mode 100644
index 00000000..f5a72156
--- /dev/null
+++ b/tests/xfs/775.out
@@ -0,0 +1,3 @@
+QA output created by 775
+Format and mount
+Try to write a bunch of stuff to the fragmented rt space


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs
  2021-09-01  0:11 [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes Darrick J. Wong
  2021-09-01  0:11 ` [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file Darrick J. Wong
@ 2021-09-01  0:11 ` Darrick J. Wong
  2021-09-05 14:35   ` Eryu Guan
  2021-09-01  0:11 ` [PATCH 3/3] xfs: test adding realtime sections to filesystem Darrick J. Wong
  2 siblings, 1 reply; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-01  0:11 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test for the following fixes:

 xfs: standardize extent size hint validation
 xfs: validate extsz hints against rt extent size when rtinherit is set
 mkfs: validate rt extent size hint when rtinherit is set

These patches fix inadequate rtextsize alignment validation of extent
size hints on directories with the rtinherit and extszinherit flags set.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/774     |   80 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/774.out |    5 +++
 tests/xfs/776     |   57 ++++++++++++++++++++++++++++++++++++++
 tests/xfs/776.out |    5 +++
 4 files changed, 147 insertions(+)
 create mode 100755 tests/xfs/774
 create mode 100644 tests/xfs/774.out
 create mode 100755 tests/xfs/776
 create mode 100644 tests/xfs/776.out


diff --git a/tests/xfs/774 b/tests/xfs/774
new file mode 100755
index 00000000..4c6bc2c9
--- /dev/null
+++ b/tests/xfs/774
@@ -0,0 +1,80 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test 774
+#
+# Regression test for kernel commits:
+#
+# 6b69e485894b ("xfs: standardize extent size hint validation")
+# 603f000b15f2 ("xfs: validate extsz hints against rt extent size when rtinherit is set")
+#
+# Regression test for xfsprogs commit:
+#
+# 1e8afffb ("mkfs: validate rt extent size hint when rtinherit is set")
+#
+# Collectively, these patches ensure that we cannot set the extent size hint on
+# a directory when the directory is configured to propagate its realtime and
+# extent size hint to newly created files when the hint size isn't aligned to
+# the size of a realtime extent.  If the patches aren't applied, the write will
+# fail and xfs_repair will say that the fs is corrupt.
+#
+. ./common/preamble
+_begin_fstest auto quick realtime
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_supported_fs generic
+_require_realtime
+_require_scratch
+
+# Check mkfs.xfs option parsing with regards to rtinherit.  XFS doesn't require
+# the realtime volume to be present to set rtinherit, so it's safe to call the
+# mkfs binary directly, in dry run mode, with exactly the parameters we want to
+# check.
+mkfs_args=(-f -N -r extsize=7b -d extszinherit=15 $SCRATCH_DEV)
+$MKFS_XFS_PROG -d rtinherit=1 "${mkfs_args[@]}" &>> $seqres.full && \
+	echo "mkfs should not succeed with heritable rtext-unaligned extent hint"
+$MKFS_XFS_PROG -d rtinherit=0 "${mkfs_args[@]}" &>> $seqres.full || \
+	echo "mkfs should succeed with uninheritable rtext-unaligned extent hint"
+
+# Move on to checking the kernel's behavior
+_scratch_mkfs -r extsize=7b | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
+cat $tmp.mkfs >> $seqres.full
+. $tmp.mkfs
+_scratch_mount
+
+test $rtextsz -ne $dbsize || _notrun "failed to set large rt extent size"
+
+# Ensure there's no extent size hint set on the directory, then set the
+# rtinherit bit on the directory to test propagation.
+$XFS_IO_PROG -c 'extsize 0' -c 'chattr +t' $SCRATCH_MNT
+
+# Now try to set an extent size hint on the directory that isn't aligned to
+# the rt extent size.
+$XFS_IO_PROG -c "extsize $((rtextsz + dbsize))" $SCRATCH_MNT 2>&1 | _filter_scratch
+$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT > $tmp.stat
+cat $tmp.stat >> $seqres.full
+grep -q 'fsxattr.xflags.*rt-inherit' $tmp.stat || \
+	echo "rtinherit didn't get set on the directory?"
+grep 'fsxattr.extsize' $tmp.stat
+
+# Propagate the hint from directory to file
+echo moo > $SCRATCH_MNT/dummy
+$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT/dummy > $tmp.stat
+cat $tmp.stat >> $seqres.full
+grep -q 'fsxattr.xflags.*realtime' $tmp.stat || \
+	echo "realtime didnt' get set on the file?"
+grep 'fsxattr.extsize' $tmp.stat
+
+# Cycle the mount to force the inode verifier to run.
+_scratch_cycle_mount
+
+# Can we still access the dummy file?
+cat $SCRATCH_MNT/dummy
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/774.out b/tests/xfs/774.out
new file mode 100644
index 00000000..767a504e
--- /dev/null
+++ b/tests/xfs/774.out
@@ -0,0 +1,5 @@
+QA output created by 774
+xfs_io: FS_IOC_FSSETXATTR SCRATCH_MNT: Invalid argument
+fsxattr.extsize = 0
+fsxattr.extsize = 0
+moo
diff --git a/tests/xfs/776 b/tests/xfs/776
new file mode 100755
index 00000000..a62da9a5
--- /dev/null
+++ b/tests/xfs/776
@@ -0,0 +1,57 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021, Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 776
+#
+# Functional test for xfsprogs commit:
+#
+# 5f062427 ("xfs_repair: validate alignment of inherited rt extent hints")
+#
+# This xfs_repair patch detects directories that are configured to propagate
+# their realtime and extent size hints to newly created realtime files when the
+# hint size isn't aligned to the size of a realtime extent.
+#
+# Since this is a test of userspace tool functionality, we don't need kernel
+# support, which in turn means that we omit _require_realtime.  Note that XFS
+# allows users to configure realtime extent size geometry and set RTINHERIT
+# flags even if the filesystem itself does not have a realtime volume attached.
+#
+. ./common/preamble
+_begin_fstest auto repair fuzzers
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_require_scratch
+
+echo "Format and mount"
+_scratch_mkfs -r extsize=7b | _filter_mkfs > $seqres.full 2>$tmp.mkfs
+cat $tmp.mkfs >> $seqres.full
+. $tmp.mkfs
+
+test $rtextsz -ne $dbsize || _notrun "failed to set large rt extent size"
+
+_scratch_mount >> $seqres.full 2>&1
+rootino=$(stat -c '%i' $SCRATCH_MNT)
+_scratch_unmount
+
+echo "Misconfigure the root directory"
+rtextsz_blks=$((rtextsz / dbsize))
+_scratch_xfs_db -x -c "inode $rootino" \
+	-c "write -d core.extsize $((rtextsz_blks + 1))" \
+	-c 'write -d core.rtinherit 1' \
+	-c 'write -d core.extszinherit 1' \
+	-c 'print' >> $seqres.full
+
+echo "Detect misconfigured directory"
+_scratch_xfs_repair -n >> $seqres.full 2>&1 && \
+	echo "repair did not catch error?"
+
+echo "Repair misconfigured directory"
+_scratch_xfs_repair >> $seqres.full 2>&1 || \
+	echo "repair did not fix error?"
+
+status=0
+exit
diff --git a/tests/xfs/776.out b/tests/xfs/776.out
new file mode 100644
index 00000000..05ea73b2
--- /dev/null
+++ b/tests/xfs/776.out
@@ -0,0 +1,5 @@
+QA output created by 776
+Format and mount
+Misconfigure the root directory
+Detect misconfigured directory
+Repair misconfigured directory


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/3] xfs: test adding realtime sections to filesystem
  2021-09-01  0:11 [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes Darrick J. Wong
  2021-09-01  0:11 ` [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file Darrick J. Wong
  2021-09-01  0:11 ` [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs Darrick J. Wong
@ 2021-09-01  0:11 ` Darrick J. Wong
  2021-09-05 14:38   ` Eryu Guan
  2 siblings, 1 reply; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-01  0:11 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

From: Darrick J. Wong <djwong@kernel.org>

Add a functional test to exercise using "xfs_growfs -e XXX -r" to add a
realtime section to a filesystem while changing the extent size.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/779     |  112 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/779.out |    2 +
 2 files changed, 114 insertions(+)
 create mode 100755 tests/xfs/779
 create mode 100644 tests/xfs/779.out


diff --git a/tests/xfs/779 b/tests/xfs/779
new file mode 100755
index 00000000..f064879c
--- /dev/null
+++ b/tests/xfs/779
@@ -0,0 +1,112 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test 779
+#
+# Test for xfs_growfs to make sure that we can add a realtime device and set
+# its extent size hint at the same time.  This also checks for the presence of
+# these patches:
+#
+#	xfs: improve FSGROWFSRT precondition checking
+#	xfs: fix an integer overflow error in xfs_growfs_rt
+#	xfs: correct the narrative around misaligned rtinherit/extszinherit dirs
+#	xfs: don't expose misaligned extszinherit hints to userspace
+#
+. ./common/preamble
+_begin_fstest auto quick realtime growfs
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_require_realtime
+_require_scratch
+
+# Format scratch fs with no realtime section.
+SCRATCH_RTDEV="" _scratch_mkfs | _filter_mkfs 2> $tmp.mkfs >> $seqres.full
+_scratch_mount
+
+# Check that there's no realtime section.
+source $tmp.mkfs
+test $rtblocks -eq 0 || echo "expected 0 rtblocks, got $rtblocks"
+
+# Compute a new rt extent size and a separate rt extent size hint to exercise
+# the code that ignores hints that aren't a multiple of the extent size.
+XFS_MAX_RTEXTSIZE=$((1024 * 1024 * 1024))
+new_rtextsz=$((rtextsz + dbsize))
+if [ $new_rtextsz -gt $XFS_MAX_RTEXTSIZE ]; then
+	new_rtextsz=$((rtextsz - dbsize))
+fi
+new_rtextsz_blocks=$(( new_rtextsz / dbsize ))
+
+new_extszhint=$((rtextsz * 2))
+if [ $new_extszhint -eq $new_rtextsz ]; then
+	new_extszhint=$((rtextsz * 3))
+fi
+
+# Set the inheritable extent size hint and rt status.
+$XFS_IO_PROG -c 'chattr +t' -c "extsize $new_extszhint" $SCRATCH_MNT
+
+# Check that the hint was set correctly
+after_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT | \
+	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
+test $after_extszhint -eq $new_extszhint || \
+	echo "expected extszhint $new_extszhint, got $after_extszhint"
+
+# Add a realtime section and change the extent size.
+echo $XFS_GROWFS_PROG -e $new_rtextsz_blocks -r $SCRATCH_MNT >> $seqres.full
+$XFS_GROWFS_PROG -e $new_rtextsz_blocks -r $SCRATCH_MNT >> $seqres.full 2> $tmp.growfs
+res=$?
+cat $tmp.growfs
+
+# If the growfs failed, skip the post-test check because the scratch fs does
+# not have SCRATCH_RTDEV configured.  If the kernel didn't support adding the
+# rt volume, skip everything else.
+if [ $res -ne 0 ]; then
+	rm -f ${RESULT_DIR}/require_scratch
+	if grep -q "Operation not supported" $tmp.growfs; then
+		_notrun "growfs not supported on rt volume"
+	fi
+fi
+
+# Now that the root directory's extsize hint is no longer aligned to the rt
+# extent size, check that we don't report it to userspace any more.
+grow_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT | \
+	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
+test $grow_extszhint -eq 0 || \
+	echo "expected post-grow extszhint 0, got $grow_extszhint"
+
+# Check that we now have rt extents.
+rtextents=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | \
+	grep 'geom.rtextents' | cut -d ' ' -f 3)
+test $rtextents -gt 0 || echo "expected rtextents > 0"
+
+# Check the new rt extent size.
+after_rtextsz_blocks=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | \
+	grep 'geom.rtextsize' | cut -d ' ' -f 3)
+test $after_rtextsz_blocks -eq $new_rtextsz_blocks || \
+	echo "expected rtextsize $new_rtextsz_blocks, got $after_rtextsz_blocks"
+
+# Create a new realtime file to prove that we can.
+echo moo > $SCRATCH_MNT/a
+sync
+$XFS_IO_PROG -c 'lsattr -v' $SCRATCH_MNT/a | \
+	cut -d ' ' -f 1 | \
+	grep -q realtime || \
+	echo "$SCRATCH_MNT/a is not a realtime file?"
+
+# Check that the root directory's hint (which was aligned before the grow and
+# misaligned after) did not propagate to the new realtime file.
+file_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT/a | \
+	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
+test $file_extszhint -eq 0 || \
+	echo "expected file extszhint 0, got $file_extszhint"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/779.out b/tests/xfs/779.out
new file mode 100644
index 00000000..1f79fae2
--- /dev/null
+++ b/tests/xfs/779.out
@@ -0,0 +1,2 @@
+QA output created by 779
+Silence is golden


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file
  2021-09-01  0:11 ` [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file Darrick J. Wong
@ 2021-09-05 14:28   ` Eryu Guan
  2021-09-13 17:37     ` Darrick J. Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eryu Guan @ 2021-09-05 14:28 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests

On Tue, Aug 31, 2021 at 05:11:30PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This is a regression test for the two realtime allocator bug fixes:
> 
> xfs: adjust rt allocation minlen when extszhint > rtextsize
> xfs: retry allocations when locality-based search fails
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks fine to me, just a few minor nits inline.

> ---
>  tests/xfs/775     |  165 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/775.out |    3 +
>  2 files changed, 168 insertions(+)
>  create mode 100755 tests/xfs/775
>  create mode 100644 tests/xfs/775.out
> 
> 
> diff --git a/tests/xfs/775 b/tests/xfs/775
> new file mode 100755
> index 00000000..e8e6b728
> --- /dev/null
> +++ b/tests/xfs/775
> @@ -0,0 +1,165 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021, Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 775
> +#
> +# Regression test for commits:
> +#
> +# 9d5e8492eee0 ("xfs: adjust rt allocation minlen when extszhint > rtextsize")
> +# 676a659b60af ("xfs: retry allocations when locality-based search fails")
> +#
> +# The first bug occurs when an extent size hint is set on a realtime file.
> +# xfs_bmapi_rtalloc adjusts the offset and length of the allocation request to
> +# try to satisfy the hint, but doesn't adjust minlen to match.  If the
> +# allocator finds free space that isn't large enough to map even a single block
> +# of the original request, bmapi_write will return ENOSPC and the write fails
> +# even though there's plenty of space.
> +#
> +# The second bug occurs when an extent size hint is set on a file, we ask to
> +# allocate blocks in an empty region immediately adjacent to a previous
> +# allocation, and the nearest available free space isn't anywhere near the
> +# previous allocation, the near allocator will give up and return ENOSPC, even
> +# if there's sufficient free realtime extents to satisfy the allocation
> +# request.
> +#
> +# Both bugs can be exploited by the same user call sequence, so here's a
> +# targeted test that runs in less time than the reproducers that are listed in
> +# the fix patches themselves.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick rw realtime
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +	cd /
> +}

Seems there's no need to override the default cleanup function?

> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here

_supported_fs xfs

> +_require_scratch
> +_require_realtime
> +_require_xfs_io_command "falloc"
> +_require_xfs_io_command "fpunch"
> +_require_test_program "punch-alternating"
> +
> +fill_rtdev()
> +{
> +	file=$1
> +
> +	filesize=`_get_available_space $SCRATCH_MNT`
> +	$XFS_IO_PROG -f -c "truncate $filesize" -c "falloc 0 $filesize" $file
> +
> +	chunks=20
> +	chunksizemb=$((filesize / chunks / 1048576))
> +	seq 1 $chunks | while read f; do
> +		echo "$((f * chunksizemb)) file size $f / 20"
> +		$XFS_IO_PROG -fc "falloc -k $(( (f - 1) * chunksizemb))m ${chunksizemb}m" $file
> +	done
> +
> +	chunks=100
> +	chunksizemb=$((filesize / chunks / 1048576))
> +	seq 80 $chunks | while read f; do
> +		echo "$((f * chunksizemb)) file size $f / $chunks"
> +		$XFS_IO_PROG -fc "falloc -k $(( (f - 1) * chunksizemb))m ${chunksizemb}m" $file
> +	done
> +
> +	filesizemb=$((filesize / 1048576))
> +	$XFS_IO_PROG -fc "falloc -k 0 ${filesizemb}m" $file
> +
> +	# Try again anyway
> +	avail=`_get_available_space $SCRATCH_MNT`
> +	$XFS_IO_PROG -fc "pwrite -S 0x65 0 $avail" ${file}
> +}
> +
> +echo "Format and mount"
> +_scratch_mkfs > $seqres.full 2>&1
> +_scratch_mount >> $seqres.full 2>&1
> +
> +# This is a test of the rt allocator; force all files to be created realtime
> +_xfs_force_bdev realtime $SCRATCH_MNT
> +
> +# Set the extent size hint larger than the realtime extent size.  This is
> +# necessary to exercise the minlen constraints on the realtime allocator.
> +fsbsize=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep geom.bsize | awk '{print $3}')
> +rtextsize_blks=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep geom.rtextsize | awk '{print $3}')
> +extsize=$((2 * rtextsize_blks * fsbsize))
> +
> +echo "rtextsize_blks=$rtextsize_blks extsize=$extsize" >> $seqres.full
> +_xfs_force_bdev realtime $SCRATCH_MNT

Already done above, could be removed?

Thanks,
Eryu

> +$XFS_IO_PROG -c "extsize $extsize" $SCRATCH_MNT
> +
> +# Compute the geometry of the test files we're going to create.  Realtime
> +# volumes are simple, which means that we can control the space allocations
> +# exactly to exploit bugs!
> +#
> +# Since this is a test of the near rt allocator, we need to set up the test to
> +# have a victim file with at least one rt extent allocated to it and enough
> +# free space to allocate at least one more rt extent at an adjacent file
> +# offset.  The free space must not be immediately adjacent to the the first
> +# extent that we allocate to the victim file, and it must not be large enough
> +# to satisfy the entire allocation request all at once.
> +#
> +# Our free space fragmentation strategy is the usual fallocate-and-punch swiss
> +# cheese file, which means the free space is split into five sections:
> +#
> +# The first will be remapped into the victim file.
> +#
> +# The second section exists to prevent the free extents from being adjacent to
> +# the first section.  It will be very large, since we allocate all the rt
> +# space.
> +#
> +# The last three sections will have every other rt extent punched out to create
> +# some free space.
> +remap_sz=$((extsize * 2))
> +required_sz=$((5 * remap_sz))
> +free_rtspace=$(_get_available_space $SCRATCH_MNT)
> +if [ $free_rtspace -lt $required_sz ]; then
> +	_notrun "Insufficient free space on rt volume.  Needed $required_sz, saw $free_rtspace."
> +fi
> +
> +# Allocate all the space on the rt volume so that we can control space
> +# allocations exactly.
> +fill_rtdev $SCRATCH_MNT/bigfile &>> $seqres.full
> +
> +# We need at least 4 remap sections to proceed
> +bigfile_sz=$(stat -c '%s' $SCRATCH_MNT/bigfile)
> +if [ $bigfile_sz -lt $required_sz ]; then
> +	_notrun "Free space control file needed $required_sz bytes, got $bigfile_sz."
> +fi
> +
> +# Remap the first remap section to a victim file.
> +$XFS_IO_PROG -c "fpunch 0 $remap_sz" $SCRATCH_MNT/bigfile
> +$XFS_IO_PROG -f -c "truncate $required_sz" -c "falloc 0 $remap_sz" $SCRATCH_MNT/victim
> +
> +# Punch out every other extent of the last two sections, to fragment free space.
> +frag_sz=$((remap_sz * 3))
> +punch_off=$((bigfile_sz - frag_sz))
> +$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / fsbsize)) -i $((rtextsize_blks * 2)) -s $rtextsize_blks
> +
> +# Make sure we have some free rtextents.
> +free_rtx=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep counts.freertx | awk '{print $3}')
> +if [ $free_rtx -eq 0 ]; then
> +	echo "Expected fragmented free rt space, found none."
> +fi
> +
> +# Try to double the amount of blocks in the victim file.  On a buggy kernel,
> +# the rt allocator will fail immediately with ENOSPC even though we left enough
> +# free space for the write will complete fully.
> +echo "Try to write a bunch of stuff to the fragmented rt space"
> +$XFS_IO_PROG -c "pwrite -S 0x63 -b $remap_sz $remap_sz $remap_sz" -c stat $SCRATCH_MNT/victim >> $seqres.full
> +
> +# The victim file should own at least two sections' worth of blocks.
> +victim_sectors=$(stat -c '%b' $SCRATCH_MNT/victim)
> +victim_space_usage=$((victim_sectors * 512))
> +expected_usage=$((remap_sz * 2))
> +
> +if [ $victim_space_usage -lt $expected_usage ]; then
> +	echo "Victim file should be using at least $expected_usage bytes, saw $victim_space_usage."
> +fi
> +
> +status=0
> +exit
> diff --git a/tests/xfs/775.out b/tests/xfs/775.out
> new file mode 100644
> index 00000000..f5a72156
> --- /dev/null
> +++ b/tests/xfs/775.out
> @@ -0,0 +1,3 @@
> +QA output created by 775
> +Format and mount
> +Try to write a bunch of stuff to the fragmented rt space

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs
  2021-09-01  0:11 ` [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs Darrick J. Wong
@ 2021-09-05 14:35   ` Eryu Guan
  2021-09-13 17:57     ` Darrick J. Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eryu Guan @ 2021-09-05 14:35 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests

On Tue, Aug 31, 2021 at 05:11:35PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This is a regression test for the following fixes:
> 
>  xfs: standardize extent size hint validation
>  xfs: validate extsz hints against rt extent size when rtinherit is set
>  mkfs: validate rt extent size hint when rtinherit is set
> 
> These patches fix inadequate rtextsize alignment validation of extent
> size hints on directories with the rtinherit and extszinherit flags set.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/774     |   80 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/774.out |    5 +++
>  tests/xfs/776     |   57 ++++++++++++++++++++++++++++++++++++++
>  tests/xfs/776.out |    5 +++
>  4 files changed, 147 insertions(+)
>  create mode 100755 tests/xfs/774
>  create mode 100644 tests/xfs/774.out
>  create mode 100755 tests/xfs/776
>  create mode 100644 tests/xfs/776.out
> 
> 
> diff --git a/tests/xfs/774 b/tests/xfs/774
> new file mode 100755
> index 00000000..4c6bc2c9
> --- /dev/null
> +++ b/tests/xfs/774
> @@ -0,0 +1,80 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test 774
> +#
> +# Regression test for kernel commits:
> +#
> +# 6b69e485894b ("xfs: standardize extent size hint validation")
> +# 603f000b15f2 ("xfs: validate extsz hints against rt extent size when rtinherit is set")
> +#
> +# Regression test for xfsprogs commit:
> +#
> +# 1e8afffb ("mkfs: validate rt extent size hint when rtinherit is set")
> +#
> +# Collectively, these patches ensure that we cannot set the extent size hint on
> +# a directory when the directory is configured to propagate its realtime and
> +# extent size hint to newly created files when the hint size isn't aligned to
> +# the size of a realtime extent.  If the patches aren't applied, the write will
> +# fail and xfs_repair will say that the fs is corrupt.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick realtime

Also add 'mkfs' group? Or we could split the mkfs test into a new test?

> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs generic

_supported_fs xfs

> +_require_realtime
> +_require_scratch
> +
> +# Check mkfs.xfs option parsing with regards to rtinherit.  XFS doesn't require
> +# the realtime volume to be present to set rtinherit, so it's safe to call the
> +# mkfs binary directly, in dry run mode, with exactly the parameters we want to
> +# check.
> +mkfs_args=(-f -N -r extsize=7b -d extszinherit=15 $SCRATCH_DEV)
> +$MKFS_XFS_PROG -d rtinherit=1 "${mkfs_args[@]}" &>> $seqres.full && \
> +	echo "mkfs should not succeed with heritable rtext-unaligned extent hint"
> +$MKFS_XFS_PROG -d rtinherit=0 "${mkfs_args[@]}" &>> $seqres.full || \
> +	echo "mkfs should succeed with uninheritable rtext-unaligned extent hint"
> +
> +# Move on to checking the kernel's behavior
> +_scratch_mkfs -r extsize=7b | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
> +cat $tmp.mkfs >> $seqres.full
> +. $tmp.mkfs
> +_scratch_mount
> +
> +test $rtextsz -ne $dbsize || _notrun "failed to set large rt extent size"

This message looks like a failure. Print which condition is not met?

> +
> +# Ensure there's no extent size hint set on the directory, then set the
> +# rtinherit bit on the directory to test propagation.
> +$XFS_IO_PROG -c 'extsize 0' -c 'chattr +t' $SCRATCH_MNT
> +
> +# Now try to set an extent size hint on the directory that isn't aligned to
> +# the rt extent size.
> +$XFS_IO_PROG -c "extsize $((rtextsz + dbsize))" $SCRATCH_MNT 2>&1 | _filter_scratch
> +$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT > $tmp.stat
> +cat $tmp.stat >> $seqres.full
> +grep -q 'fsxattr.xflags.*rt-inherit' $tmp.stat || \
> +	echo "rtinherit didn't get set on the directory?"
> +grep 'fsxattr.extsize' $tmp.stat
> +
> +# Propagate the hint from directory to file
> +echo moo > $SCRATCH_MNT/dummy
> +$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT/dummy > $tmp.stat
> +cat $tmp.stat >> $seqres.full
> +grep -q 'fsxattr.xflags.*realtime' $tmp.stat || \
> +	echo "realtime didnt' get set on the file?"
> +grep 'fsxattr.extsize' $tmp.stat
> +
> +# Cycle the mount to force the inode verifier to run.
> +_scratch_cycle_mount
> +
> +# Can we still access the dummy file?
> +cat $SCRATCH_MNT/dummy
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/774.out b/tests/xfs/774.out
> new file mode 100644
> index 00000000..767a504e
> --- /dev/null
> +++ b/tests/xfs/774.out
> @@ -0,0 +1,5 @@
> +QA output created by 774
> +xfs_io: FS_IOC_FSSETXATTR SCRATCH_MNT: Invalid argument
> +fsxattr.extsize = 0
> +fsxattr.extsize = 0
> +moo
> diff --git a/tests/xfs/776 b/tests/xfs/776
> new file mode 100755
> index 00000000..a62da9a5
> --- /dev/null
> +++ b/tests/xfs/776
> @@ -0,0 +1,57 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021, Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 776
> +#
> +# Functional test for xfsprogs commit:
> +#
> +# 5f062427 ("xfs_repair: validate alignment of inherited rt extent hints")
> +#
> +# This xfs_repair patch detects directories that are configured to propagate
> +# their realtime and extent size hints to newly created realtime files when the
> +# hint size isn't aligned to the size of a realtime extent.
> +#
> +# Since this is a test of userspace tool functionality, we don't need kernel
> +# support, which in turn means that we omit _require_realtime.  Note that XFS
> +# allows users to configure realtime extent size geometry and set RTINHERIT
> +# flags even if the filesystem itself does not have a realtime volume attached.
> +#
> +. ./common/preamble
> +_begin_fstest auto repair fuzzers
> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here

_supported_fs xfs

> +_require_scratch
> +
> +echo "Format and mount"
> +_scratch_mkfs -r extsize=7b | _filter_mkfs > $seqres.full 2>$tmp.mkfs
> +cat $tmp.mkfs >> $seqres.full
> +. $tmp.mkfs
> +
> +test $rtextsz -ne $dbsize || _notrun "failed to set large rt extent size"

Same here.

Thanks,
Eryu

> +
> +_scratch_mount >> $seqres.full 2>&1
> +rootino=$(stat -c '%i' $SCRATCH_MNT)
> +_scratch_unmount
> +
> +echo "Misconfigure the root directory"
> +rtextsz_blks=$((rtextsz / dbsize))
> +_scratch_xfs_db -x -c "inode $rootino" \
> +	-c "write -d core.extsize $((rtextsz_blks + 1))" \
> +	-c 'write -d core.rtinherit 1' \
> +	-c 'write -d core.extszinherit 1' \
> +	-c 'print' >> $seqres.full
> +
> +echo "Detect misconfigured directory"
> +_scratch_xfs_repair -n >> $seqres.full 2>&1 && \
> +	echo "repair did not catch error?"
> +
> +echo "Repair misconfigured directory"
> +_scratch_xfs_repair >> $seqres.full 2>&1 || \
> +	echo "repair did not fix error?"
> +
> +status=0
> +exit
> diff --git a/tests/xfs/776.out b/tests/xfs/776.out
> new file mode 100644
> index 00000000..05ea73b2
> --- /dev/null
> +++ b/tests/xfs/776.out
> @@ -0,0 +1,5 @@
> +QA output created by 776
> +Format and mount
> +Misconfigure the root directory
> +Detect misconfigured directory
> +Repair misconfigured directory

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] xfs: test adding realtime sections to filesystem
  2021-09-01  0:11 ` [PATCH 3/3] xfs: test adding realtime sections to filesystem Darrick J. Wong
@ 2021-09-05 14:38   ` Eryu Guan
  2021-09-13 18:02     ` Darrick J. Wong
  0 siblings, 1 reply; 11+ messages in thread
From: Eryu Guan @ 2021-09-05 14:38 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests

On Tue, Aug 31, 2021 at 05:11:41PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Add a functional test to exercise using "xfs_growfs -e XXX -r" to add a
> realtime section to a filesystem while changing the extent size.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/779     |  112 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/779.out |    2 +
>  2 files changed, 114 insertions(+)
>  create mode 100755 tests/xfs/779
>  create mode 100644 tests/xfs/779.out
> 
> 
> diff --git a/tests/xfs/779 b/tests/xfs/779
> new file mode 100755
> index 00000000..f064879c
> --- /dev/null
> +++ b/tests/xfs/779
> @@ -0,0 +1,112 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test 779
> +#
> +# Test for xfs_growfs to make sure that we can add a realtime device and set
> +# its extent size hint at the same time.  This also checks for the presence of
> +# these patches:
> +#
> +#	xfs: improve FSGROWFSRT precondition checking
> +#	xfs: fix an integer overflow error in xfs_growfs_rt
> +#	xfs: correct the narrative around misaligned rtinherit/extszinherit dirs
> +#	xfs: don't expose misaligned extszinherit hints to userspace

The patches are now merged, include their commit ids as well?

> +#
> +. ./common/preamble
> +_begin_fstest auto quick realtime growfs
> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic

_supported_fs xfs

> +_require_realtime
> +_require_scratch
> +
> +# Format scratch fs with no realtime section.
> +SCRATCH_RTDEV="" _scratch_mkfs | _filter_mkfs 2> $tmp.mkfs >> $seqres.full
> +_scratch_mount
> +
> +# Check that there's no realtime section.
> +source $tmp.mkfs
> +test $rtblocks -eq 0 || echo "expected 0 rtblocks, got $rtblocks"
> +
> +# Compute a new rt extent size and a separate rt extent size hint to exercise
> +# the code that ignores hints that aren't a multiple of the extent size.
> +XFS_MAX_RTEXTSIZE=$((1024 * 1024 * 1024))
> +new_rtextsz=$((rtextsz + dbsize))
> +if [ $new_rtextsz -gt $XFS_MAX_RTEXTSIZE ]; then
> +	new_rtextsz=$((rtextsz - dbsize))
> +fi
> +new_rtextsz_blocks=$(( new_rtextsz / dbsize ))
> +
> +new_extszhint=$((rtextsz * 2))
> +if [ $new_extszhint -eq $new_rtextsz ]; then
> +	new_extszhint=$((rtextsz * 3))
> +fi
> +
> +# Set the inheritable extent size hint and rt status.
> +$XFS_IO_PROG -c 'chattr +t' -c "extsize $new_extszhint" $SCRATCH_MNT
> +
> +# Check that the hint was set correctly
> +after_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT | \
> +	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
> +test $after_extszhint -eq $new_extszhint || \
> +	echo "expected extszhint $new_extszhint, got $after_extszhint"
> +
> +# Add a realtime section and change the extent size.
> +echo $XFS_GROWFS_PROG -e $new_rtextsz_blocks -r $SCRATCH_MNT >> $seqres.full
> +$XFS_GROWFS_PROG -e $new_rtextsz_blocks -r $SCRATCH_MNT >> $seqres.full 2> $tmp.growfs
> +res=$?
> +cat $tmp.growfs
> +
> +# If the growfs failed, skip the post-test check because the scratch fs does
> +# not have SCRATCH_RTDEV configured.  If the kernel didn't support adding the
> +# rt volume, skip everything else.
> +if [ $res -ne 0 ]; then
> +	rm -f ${RESULT_DIR}/require_scratch
> +	if grep -q "Operation not supported" $tmp.growfs; then
> +		_notrun "growfs not supported on rt volume"
> +	fi
> +fi
> +
> +# Now that the root directory's extsize hint is no longer aligned to the rt
> +# extent size, check that we don't report it to userspace any more.
> +grow_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT | \
> +	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
> +test $grow_extszhint -eq 0 || \
> +	echo "expected post-grow extszhint 0, got $grow_extszhint"
> +
> +# Check that we now have rt extents.
> +rtextents=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | \
> +	grep 'geom.rtextents' | cut -d ' ' -f 3)
> +test $rtextents -gt 0 || echo "expected rtextents > 0"
> +
> +# Check the new rt extent size.
> +after_rtextsz_blocks=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | \
> +	grep 'geom.rtextsize' | cut -d ' ' -f 3)
> +test $after_rtextsz_blocks -eq $new_rtextsz_blocks || \
> +	echo "expected rtextsize $new_rtextsz_blocks, got $after_rtextsz_blocks"
> +
> +# Create a new realtime file to prove that we can.
> +echo moo > $SCRATCH_MNT/a
> +sync

A global sync is a bit heavy, fsync the file or syncfs $SCRATCH_MNT ?

Thanks,
Eryu

> +$XFS_IO_PROG -c 'lsattr -v' $SCRATCH_MNT/a | \
> +	cut -d ' ' -f 1 | \
> +	grep -q realtime || \
> +	echo "$SCRATCH_MNT/a is not a realtime file?"
> +
> +# Check that the root directory's hint (which was aligned before the grow and
> +# misaligned after) did not propagate to the new realtime file.
> +file_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT/a | \
> +	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
> +test $file_extszhint -eq 0 || \
> +	echo "expected file extszhint 0, got $file_extszhint"
> +
> +# success, all done
> +echo Silence is golden
> +status=0
> +exit
> diff --git a/tests/xfs/779.out b/tests/xfs/779.out
> new file mode 100644
> index 00000000..1f79fae2
> --- /dev/null
> +++ b/tests/xfs/779.out
> @@ -0,0 +1,2 @@
> +QA output created by 779
> +Silence is golden

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file
  2021-09-05 14:28   ` Eryu Guan
@ 2021-09-13 17:37     ` Darrick J. Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-13 17:37 UTC (permalink / raw)
  To: Eryu Guan; +Cc: guaneryu, linux-xfs, fstests

On Sun, Sep 05, 2021 at 10:28:16PM +0800, Eryu Guan wrote:
> On Tue, Aug 31, 2021 at 05:11:30PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > This is a regression test for the two realtime allocator bug fixes:
> > 
> > xfs: adjust rt allocation minlen when extszhint > rtextsize
> > xfs: retry allocations when locality-based search fails
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> 
> Looks fine to me, just a few minor nits inline.
> 
> > ---
> >  tests/xfs/775     |  165 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/775.out |    3 +
> >  2 files changed, 168 insertions(+)
> >  create mode 100755 tests/xfs/775
> >  create mode 100644 tests/xfs/775.out
> > 
> > 
> > diff --git a/tests/xfs/775 b/tests/xfs/775
> > new file mode 100755
> > index 00000000..e8e6b728
> > --- /dev/null
> > +++ b/tests/xfs/775
> > @@ -0,0 +1,165 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021, Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 775
> > +#
> > +# Regression test for commits:
> > +#
> > +# 9d5e8492eee0 ("xfs: adjust rt allocation minlen when extszhint > rtextsize")
> > +# 676a659b60af ("xfs: retry allocations when locality-based search fails")
> > +#
> > +# The first bug occurs when an extent size hint is set on a realtime file.
> > +# xfs_bmapi_rtalloc adjusts the offset and length of the allocation request to
> > +# try to satisfy the hint, but doesn't adjust minlen to match.  If the
> > +# allocator finds free space that isn't large enough to map even a single block
> > +# of the original request, bmapi_write will return ENOSPC and the write fails
> > +# even though there's plenty of space.
> > +#
> > +# The second bug occurs when an extent size hint is set on a file, we ask to
> > +# allocate blocks in an empty region immediately adjacent to a previous
> > +# allocation, and the nearest available free space isn't anywhere near the
> > +# previous allocation, the near allocator will give up and return ENOSPC, even
> > +# if there's sufficient free realtime extents to satisfy the allocation
> > +# request.
> > +#
> > +# Both bugs can be exploited by the same user call sequence, so here's a
> > +# targeted test that runs in less time than the reproducers that are listed in
> > +# the fix patches themselves.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick rw realtime
> > +
> > +# Override the default cleanup function.
> > +_cleanup()
> > +{
> > +	cd /
> > +}
> 
> Seems there's no need to override the default cleanup function?

Yeah.  I'll clobber it.

> > +
> > +# Import common functions.
> > +. ./common/filter
> > +
> > +# real QA test starts here
> 
> _supported_fs xfs

Fixed.

> > +_require_scratch
> > +_require_realtime
> > +_require_xfs_io_command "falloc"
> > +_require_xfs_io_command "fpunch"
> > +_require_test_program "punch-alternating"
> > +
> > +fill_rtdev()
> > +{
> > +	file=$1
> > +
> > +	filesize=`_get_available_space $SCRATCH_MNT`
> > +	$XFS_IO_PROG -f -c "truncate $filesize" -c "falloc 0 $filesize" $file
> > +
> > +	chunks=20
> > +	chunksizemb=$((filesize / chunks / 1048576))
> > +	seq 1 $chunks | while read f; do
> > +		echo "$((f * chunksizemb)) file size $f / 20"
> > +		$XFS_IO_PROG -fc "falloc -k $(( (f - 1) * chunksizemb))m ${chunksizemb}m" $file
> > +	done
> > +
> > +	chunks=100
> > +	chunksizemb=$((filesize / chunks / 1048576))
> > +	seq 80 $chunks | while read f; do
> > +		echo "$((f * chunksizemb)) file size $f / $chunks"
> > +		$XFS_IO_PROG -fc "falloc -k $(( (f - 1) * chunksizemb))m ${chunksizemb}m" $file
> > +	done
> > +
> > +	filesizemb=$((filesize / 1048576))
> > +	$XFS_IO_PROG -fc "falloc -k 0 ${filesizemb}m" $file
> > +
> > +	# Try again anyway
> > +	avail=`_get_available_space $SCRATCH_MNT`
> > +	$XFS_IO_PROG -fc "pwrite -S 0x65 0 $avail" ${file}
> > +}
> > +
> > +echo "Format and mount"
> > +_scratch_mkfs > $seqres.full 2>&1
> > +_scratch_mount >> $seqres.full 2>&1
> > +
> > +# This is a test of the rt allocator; force all files to be created realtime
> > +_xfs_force_bdev realtime $SCRATCH_MNT
> > +
> > +# Set the extent size hint larger than the realtime extent size.  This is
> > +# necessary to exercise the minlen constraints on the realtime allocator.
> > +fsbsize=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep geom.bsize | awk '{print $3}')
> > +rtextsize_blks=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep geom.rtextsize | awk '{print $3}')
> > +extsize=$((2 * rtextsize_blks * fsbsize))
> > +
> > +echo "rtextsize_blks=$rtextsize_blks extsize=$extsize" >> $seqres.full
> > +_xfs_force_bdev realtime $SCRATCH_MNT
> 
> Already done above, could be removed?

Yup, fixed.

--D

> Thanks,
> Eryu
> 
> > +$XFS_IO_PROG -c "extsize $extsize" $SCRATCH_MNT
> > +
> > +# Compute the geometry of the test files we're going to create.  Realtime
> > +# volumes are simple, which means that we can control the space allocations
> > +# exactly to exploit bugs!
> > +#
> > +# Since this is a test of the near rt allocator, we need to set up the test to
> > +# have a victim file with at least one rt extent allocated to it and enough
> > +# free space to allocate at least one more rt extent at an adjacent file
> > +# offset.  The free space must not be immediately adjacent to the the first
> > +# extent that we allocate to the victim file, and it must not be large enough
> > +# to satisfy the entire allocation request all at once.
> > +#
> > +# Our free space fragmentation strategy is the usual fallocate-and-punch swiss
> > +# cheese file, which means the free space is split into five sections:
> > +#
> > +# The first will be remapped into the victim file.
> > +#
> > +# The second section exists to prevent the free extents from being adjacent to
> > +# the first section.  It will be very large, since we allocate all the rt
> > +# space.
> > +#
> > +# The last three sections will have every other rt extent punched out to create
> > +# some free space.
> > +remap_sz=$((extsize * 2))
> > +required_sz=$((5 * remap_sz))
> > +free_rtspace=$(_get_available_space $SCRATCH_MNT)
> > +if [ $free_rtspace -lt $required_sz ]; then
> > +	_notrun "Insufficient free space on rt volume.  Needed $required_sz, saw $free_rtspace."
> > +fi
> > +
> > +# Allocate all the space on the rt volume so that we can control space
> > +# allocations exactly.
> > +fill_rtdev $SCRATCH_MNT/bigfile &>> $seqres.full
> > +
> > +# We need at least 4 remap sections to proceed
> > +bigfile_sz=$(stat -c '%s' $SCRATCH_MNT/bigfile)
> > +if [ $bigfile_sz -lt $required_sz ]; then
> > +	_notrun "Free space control file needed $required_sz bytes, got $bigfile_sz."
> > +fi
> > +
> > +# Remap the first remap section to a victim file.
> > +$XFS_IO_PROG -c "fpunch 0 $remap_sz" $SCRATCH_MNT/bigfile
> > +$XFS_IO_PROG -f -c "truncate $required_sz" -c "falloc 0 $remap_sz" $SCRATCH_MNT/victim
> > +
> > +# Punch out every other extent of the last two sections, to fragment free space.
> > +frag_sz=$((remap_sz * 3))
> > +punch_off=$((bigfile_sz - frag_sz))
> > +$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / fsbsize)) -i $((rtextsize_blks * 2)) -s $rtextsize_blks
> > +
> > +# Make sure we have some free rtextents.
> > +free_rtx=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep counts.freertx | awk '{print $3}')
> > +if [ $free_rtx -eq 0 ]; then
> > +	echo "Expected fragmented free rt space, found none."
> > +fi
> > +
> > +# Try to double the amount of blocks in the victim file.  On a buggy kernel,
> > +# the rt allocator will fail immediately with ENOSPC even though we left enough
> > +# free space for the write will complete fully.
> > +echo "Try to write a bunch of stuff to the fragmented rt space"
> > +$XFS_IO_PROG -c "pwrite -S 0x63 -b $remap_sz $remap_sz $remap_sz" -c stat $SCRATCH_MNT/victim >> $seqres.full
> > +
> > +# The victim file should own at least two sections' worth of blocks.
> > +victim_sectors=$(stat -c '%b' $SCRATCH_MNT/victim)
> > +victim_space_usage=$((victim_sectors * 512))
> > +expected_usage=$((remap_sz * 2))
> > +
> > +if [ $victim_space_usage -lt $expected_usage ]; then
> > +	echo "Victim file should be using at least $expected_usage bytes, saw $victim_space_usage."
> > +fi
> > +
> > +status=0
> > +exit
> > diff --git a/tests/xfs/775.out b/tests/xfs/775.out
> > new file mode 100644
> > index 00000000..f5a72156
> > --- /dev/null
> > +++ b/tests/xfs/775.out
> > @@ -0,0 +1,3 @@
> > +QA output created by 775
> > +Format and mount
> > +Try to write a bunch of stuff to the fragmented rt space

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs
  2021-09-05 14:35   ` Eryu Guan
@ 2021-09-13 17:57     ` Darrick J. Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-13 17:57 UTC (permalink / raw)
  To: Eryu Guan; +Cc: guaneryu, linux-xfs, fstests

On Sun, Sep 05, 2021 at 10:35:57PM +0800, Eryu Guan wrote:
> On Tue, Aug 31, 2021 at 05:11:35PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > This is a regression test for the following fixes:
> > 
> >  xfs: standardize extent size hint validation
> >  xfs: validate extsz hints against rt extent size when rtinherit is set
> >  mkfs: validate rt extent size hint when rtinherit is set
> > 
> > These patches fix inadequate rtextsize alignment validation of extent
> > size hints on directories with the rtinherit and extszinherit flags set.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/774     |   80 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/774.out |    5 +++
> >  tests/xfs/776     |   57 ++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/776.out |    5 +++
> >  4 files changed, 147 insertions(+)
> >  create mode 100755 tests/xfs/774
> >  create mode 100644 tests/xfs/774.out
> >  create mode 100755 tests/xfs/776
> >  create mode 100644 tests/xfs/776.out
> > 
> > 
> > diff --git a/tests/xfs/774 b/tests/xfs/774
> > new file mode 100755
> > index 00000000..4c6bc2c9
> > --- /dev/null
> > +++ b/tests/xfs/774
> > @@ -0,0 +1,80 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test 774
> > +#
> > +# Regression test for kernel commits:
> > +#
> > +# 6b69e485894b ("xfs: standardize extent size hint validation")
> > +# 603f000b15f2 ("xfs: validate extsz hints against rt extent size when rtinherit is set")
> > +#
> > +# Regression test for xfsprogs commit:
> > +#
> > +# 1e8afffb ("mkfs: validate rt extent size hint when rtinherit is set")
> > +#
> > +# Collectively, these patches ensure that we cannot set the extent size hint on
> > +# a directory when the directory is configured to propagate its realtime and
> > +# extent size hint to newly created files when the hint size isn't aligned to
> > +# the size of a realtime extent.  If the patches aren't applied, the write will
> > +# fail and xfs_repair will say that the fs is corrupt.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick realtime
> 
> Also add 'mkfs' group?

I'll add it to the group list.

> Or we could split the mkfs test into a new test?

I could, but...

> > +
> > +# Import common functions.
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> 
> _supported_fs xfs
> 
> > +_require_realtime
> > +_require_scratch
> > +
> > +# Check mkfs.xfs option parsing with regards to rtinherit.  XFS doesn't require
> > +# the realtime volume to be present to set rtinherit, so it's safe to call the
> > +# mkfs binary directly, in dry run mode, with exactly the parameters we want to
> > +# check.
> > +mkfs_args=(-f -N -r extsize=7b -d extszinherit=15 $SCRATCH_DEV)
> > +$MKFS_XFS_PROG -d rtinherit=1 "${mkfs_args[@]}" &>> $seqres.full && \
> > +	echo "mkfs should not succeed with heritable rtext-unaligned extent hint"
> > +$MKFS_XFS_PROG -d rtinherit=0 "${mkfs_args[@]}" &>> $seqres.full || \
> > +	echo "mkfs should succeed with uninheritable rtext-unaligned extent hint"

...these two lines are all that's needed to test the mkfs patch.  I've
simultaneoulsy gotten criticism in the past for (a) running up the
number of tests in the test suite (and hence increasing fstests runtime)
and (b) cramming too many small things into a test case, so I really
don't know where the dividing line is anymore.

Usually this means I cram everything into as few tests as possible,
except for tests that require out of band updates to ondisk metadata
(e.g. fuzzers), tests that can crash the kernel, or anything else that
doesn't "feel" like it fits the usual test behavior ... and then split
things on demand.

That was a longwinded way to say that I'll split the mkfs part into a
two-line test if you insist.

> > +
> > +# Move on to checking the kernel's behavior
> > +_scratch_mkfs -r extsize=7b | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
> > +cat $tmp.mkfs >> $seqres.full
> > +. $tmp.mkfs
> > +_scratch_mount
> > +
> > +test $rtextsz -ne $dbsize || _notrun "failed to set large rt extent size"
> 
> This message looks like a failure. Print which condition is not met?

Ok, I'll change it to:

_notrun "cannot set rt extent size ($rtextsz) larger than fs block size ($dbsize)"

> > +
> > +# Ensure there's no extent size hint set on the directory, then set the
> > +# rtinherit bit on the directory to test propagation.
> > +$XFS_IO_PROG -c 'extsize 0' -c 'chattr +t' $SCRATCH_MNT
> > +
> > +# Now try to set an extent size hint on the directory that isn't aligned to
> > +# the rt extent size.
> > +$XFS_IO_PROG -c "extsize $((rtextsz + dbsize))" $SCRATCH_MNT 2>&1 | _filter_scratch
> > +$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT > $tmp.stat
> > +cat $tmp.stat >> $seqres.full
> > +grep -q 'fsxattr.xflags.*rt-inherit' $tmp.stat || \
> > +	echo "rtinherit didn't get set on the directory?"
> > +grep 'fsxattr.extsize' $tmp.stat
> > +
> > +# Propagate the hint from directory to file
> > +echo moo > $SCRATCH_MNT/dummy
> > +$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT/dummy > $tmp.stat
> > +cat $tmp.stat >> $seqres.full
> > +grep -q 'fsxattr.xflags.*realtime' $tmp.stat || \
> > +	echo "realtime didnt' get set on the file?"
> > +grep 'fsxattr.extsize' $tmp.stat
> > +
> > +# Cycle the mount to force the inode verifier to run.
> > +_scratch_cycle_mount
> > +
> > +# Can we still access the dummy file?
> > +cat $SCRATCH_MNT/dummy
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/xfs/774.out b/tests/xfs/774.out
> > new file mode 100644
> > index 00000000..767a504e
> > --- /dev/null
> > +++ b/tests/xfs/774.out
> > @@ -0,0 +1,5 @@
> > +QA output created by 774
> > +xfs_io: FS_IOC_FSSETXATTR SCRATCH_MNT: Invalid argument
> > +fsxattr.extsize = 0
> > +fsxattr.extsize = 0
> > +moo
> > diff --git a/tests/xfs/776 b/tests/xfs/776
> > new file mode 100755
> > index 00000000..a62da9a5
> > --- /dev/null
> > +++ b/tests/xfs/776
> > @@ -0,0 +1,57 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021, Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 776
> > +#
> > +# Functional test for xfsprogs commit:
> > +#
> > +# 5f062427 ("xfs_repair: validate alignment of inherited rt extent hints")
> > +#
> > +# This xfs_repair patch detects directories that are configured to propagate
> > +# their realtime and extent size hints to newly created realtime files when the
> > +# hint size isn't aligned to the size of a realtime extent.
> > +#
> > +# Since this is a test of userspace tool functionality, we don't need kernel
> > +# support, which in turn means that we omit _require_realtime.  Note that XFS
> > +# allows users to configure realtime extent size geometry and set RTINHERIT
> > +# flags even if the filesystem itself does not have a realtime volume attached.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto repair fuzzers
> > +
> > +# Import common functions.
> > +. ./common/filter
> > +
> > +# real QA test starts here
> 
> _supported_fs xfs
> 
> > +_require_scratch
> > +
> > +echo "Format and mount"
> > +_scratch_mkfs -r extsize=7b | _filter_mkfs > $seqres.full 2>$tmp.mkfs
> > +cat $tmp.mkfs >> $seqres.full
> > +. $tmp.mkfs
> > +
> > +test $rtextsz -ne $dbsize || _notrun "failed to set large rt extent size"
> 
> Same here.

Ok, fixed.  Sorry that a lot of my tests seem to omit _supported_fs;
I'll go check the rest of my tree before I resubmit.

--D

> Thanks,
> Eryu
> 
> > +
> > +_scratch_mount >> $seqres.full 2>&1
> > +rootino=$(stat -c '%i' $SCRATCH_MNT)
> > +_scratch_unmount
> > +
> > +echo "Misconfigure the root directory"
> > +rtextsz_blks=$((rtextsz / dbsize))
> > +_scratch_xfs_db -x -c "inode $rootino" \
> > +	-c "write -d core.extsize $((rtextsz_blks + 1))" \
> > +	-c 'write -d core.rtinherit 1' \
> > +	-c 'write -d core.extszinherit 1' \
> > +	-c 'print' >> $seqres.full
> > +
> > +echo "Detect misconfigured directory"
> > +_scratch_xfs_repair -n >> $seqres.full 2>&1 && \
> > +	echo "repair did not catch error?"
> > +
> > +echo "Repair misconfigured directory"
> > +_scratch_xfs_repair >> $seqres.full 2>&1 || \
> > +	echo "repair did not fix error?"
> > +
> > +status=0
> > +exit
> > diff --git a/tests/xfs/776.out b/tests/xfs/776.out
> > new file mode 100644
> > index 00000000..05ea73b2
> > --- /dev/null
> > +++ b/tests/xfs/776.out
> > @@ -0,0 +1,5 @@
> > +QA output created by 776
> > +Format and mount
> > +Misconfigure the root directory
> > +Detect misconfigured directory
> > +Repair misconfigured directory

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH 3/3] xfs: test adding realtime sections to filesystem
  2021-09-05 14:38   ` Eryu Guan
@ 2021-09-13 18:02     ` Darrick J. Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Darrick J. Wong @ 2021-09-13 18:02 UTC (permalink / raw)
  To: Eryu Guan; +Cc: guaneryu, linux-xfs, fstests

On Sun, Sep 05, 2021 at 10:38:37PM +0800, Eryu Guan wrote:
> On Tue, Aug 31, 2021 at 05:11:41PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Add a functional test to exercise using "xfs_growfs -e XXX -r" to add a
> > realtime section to a filesystem while changing the extent size.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/779     |  112 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/779.out |    2 +
> >  2 files changed, 114 insertions(+)
> >  create mode 100755 tests/xfs/779
> >  create mode 100644 tests/xfs/779.out
> > 
> > 
> > diff --git a/tests/xfs/779 b/tests/xfs/779
> > new file mode 100755
> > index 00000000..f064879c
> > --- /dev/null
> > +++ b/tests/xfs/779
> > @@ -0,0 +1,112 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test 779
> > +#
> > +# Test for xfs_growfs to make sure that we can add a realtime device and set
> > +# its extent size hint at the same time.  This also checks for the presence of
> > +# these patches:
> > +#
> > +#	xfs: improve FSGROWFSRT precondition checking
> > +#	xfs: fix an integer overflow error in xfs_growfs_rt
> > +#	xfs: correct the narrative around misaligned rtinherit/extszinherit dirs
> > +#	xfs: don't expose misaligned extszinherit hints to userspace
> 
> The patches are now merged, include their commit ids as well?

Oops, sorry, this one slipped through when I tried to correct all that. :(

> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick realtime growfs
> > +
> > +# Import common functions.
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +
> > +# Modify as appropriate.
> > +_supported_fs generic
> 
> _supported_fs xfs

Fixed, thanks.

> > +_require_realtime
> > +_require_scratch
> > +
> > +# Format scratch fs with no realtime section.
> > +SCRATCH_RTDEV="" _scratch_mkfs | _filter_mkfs 2> $tmp.mkfs >> $seqres.full
> > +_scratch_mount
> > +
> > +# Check that there's no realtime section.
> > +source $tmp.mkfs
> > +test $rtblocks -eq 0 || echo "expected 0 rtblocks, got $rtblocks"
> > +
> > +# Compute a new rt extent size and a separate rt extent size hint to exercise
> > +# the code that ignores hints that aren't a multiple of the extent size.
> > +XFS_MAX_RTEXTSIZE=$((1024 * 1024 * 1024))
> > +new_rtextsz=$((rtextsz + dbsize))
> > +if [ $new_rtextsz -gt $XFS_MAX_RTEXTSIZE ]; then
> > +	new_rtextsz=$((rtextsz - dbsize))
> > +fi
> > +new_rtextsz_blocks=$(( new_rtextsz / dbsize ))
> > +
> > +new_extszhint=$((rtextsz * 2))
> > +if [ $new_extszhint -eq $new_rtextsz ]; then
> > +	new_extszhint=$((rtextsz * 3))
> > +fi
> > +
> > +# Set the inheritable extent size hint and rt status.
> > +$XFS_IO_PROG -c 'chattr +t' -c "extsize $new_extszhint" $SCRATCH_MNT
> > +
> > +# Check that the hint was set correctly
> > +after_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT | \
> > +	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
> > +test $after_extszhint -eq $new_extszhint || \
> > +	echo "expected extszhint $new_extszhint, got $after_extszhint"
> > +
> > +# Add a realtime section and change the extent size.
> > +echo $XFS_GROWFS_PROG -e $new_rtextsz_blocks -r $SCRATCH_MNT >> $seqres.full
> > +$XFS_GROWFS_PROG -e $new_rtextsz_blocks -r $SCRATCH_MNT >> $seqres.full 2> $tmp.growfs
> > +res=$?
> > +cat $tmp.growfs
> > +
> > +# If the growfs failed, skip the post-test check because the scratch fs does
> > +# not have SCRATCH_RTDEV configured.  If the kernel didn't support adding the
> > +# rt volume, skip everything else.
> > +if [ $res -ne 0 ]; then
> > +	rm -f ${RESULT_DIR}/require_scratch
> > +	if grep -q "Operation not supported" $tmp.growfs; then
> > +		_notrun "growfs not supported on rt volume"
> > +	fi
> > +fi
> > +
> > +# Now that the root directory's extsize hint is no longer aligned to the rt
> > +# extent size, check that we don't report it to userspace any more.
> > +grow_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT | \
> > +	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
> > +test $grow_extszhint -eq 0 || \
> > +	echo "expected post-grow extszhint 0, got $grow_extszhint"
> > +
> > +# Check that we now have rt extents.
> > +rtextents=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | \
> > +	grep 'geom.rtextents' | cut -d ' ' -f 3)
> > +test $rtextents -gt 0 || echo "expected rtextents > 0"
> > +
> > +# Check the new rt extent size.
> > +after_rtextsz_blocks=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | \
> > +	grep 'geom.rtextsize' | cut -d ' ' -f 3)
> > +test $after_rtextsz_blocks -eq $new_rtextsz_blocks || \
> > +	echo "expected rtextsize $new_rtextsz_blocks, got $after_rtextsz_blocks"
> > +
> > +# Create a new realtime file to prove that we can.
> > +echo moo > $SCRATCH_MNT/a
> > +sync
> 
> A global sync is a bit heavy, fsync the file or syncfs $SCRATCH_MNT ?

Ok, I'll make that "sync -f $SCRATCH_MNT".

--D

> Thanks,
> Eryu
> 
> > +$XFS_IO_PROG -c 'lsattr -v' $SCRATCH_MNT/a | \
> > +	cut -d ' ' -f 1 | \
> > +	grep -q realtime || \
> > +	echo "$SCRATCH_MNT/a is not a realtime file?"
> > +
> > +# Check that the root directory's hint (which was aligned before the grow and
> > +# misaligned after) did not propagate to the new realtime file.
> > +file_extszhint=$($XFS_IO_PROG -c 'stat' $SCRATCH_MNT/a | \
> > +	grep 'fsxattr.extsize' | cut -d ' ' -f 3)
> > +test $file_extszhint -eq 0 || \
> > +	echo "expected file extszhint 0, got $file_extszhint"
> > +
> > +# success, all done
> > +echo Silence is golden
> > +status=0
> > +exit
> > diff --git a/tests/xfs/779.out b/tests/xfs/779.out
> > new file mode 100644
> > index 00000000..1f79fae2
> > --- /dev/null
> > +++ b/tests/xfs/779.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 779
> > +Silence is golden

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes
@ 2021-07-28  0:10 Darrick J. Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Darrick J. Wong @ 2021-07-28  0:10 UTC (permalink / raw)
  To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan

Hi all,

Add regression tests to trigger some bugs in the realtime allocator that
were fixed in kernel 5.13 and 5.14.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-fixes
---
 tests/xfs/774     |   78 +++++++++++++++++++++++++
 tests/xfs/774.out |    5 ++
 tests/xfs/775     |  165 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/775.out |    3 +
 tests/xfs/776     |   57 ++++++++++++++++++
 tests/xfs/776.out |    5 ++
 tests/xfs/779     |  112 ++++++++++++++++++++++++++++++++++++
 tests/xfs/779.out |    2 +
 8 files changed, 427 insertions(+)
 create mode 100755 tests/xfs/774
 create mode 100644 tests/xfs/774.out
 create mode 100755 tests/xfs/775
 create mode 100644 tests/xfs/775.out
 create mode 100755 tests/xfs/776
 create mode 100644 tests/xfs/776.out
 create mode 100755 tests/xfs/779
 create mode 100644 tests/xfs/779.out


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-09-13 18:02 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-01  0:11 [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes Darrick J. Wong
2021-09-01  0:11 ` [PATCH 1/3] xfs: test fsx with extent size hints set on a realtime file Darrick J. Wong
2021-09-05 14:28   ` Eryu Guan
2021-09-13 17:37     ` Darrick J. Wong
2021-09-01  0:11 ` [PATCH 2/3] xfs: test correct propagation of rt extent size hints on rtinherit dirs Darrick J. Wong
2021-09-05 14:35   ` Eryu Guan
2021-09-13 17:57     ` Darrick J. Wong
2021-09-01  0:11 ` [PATCH 3/3] xfs: test adding realtime sections to filesystem Darrick J. Wong
2021-09-05 14:38   ` Eryu Guan
2021-09-13 18:02     ` Darrick J. Wong
  -- strict thread matches above, loose matches on Subject: below --
2021-07-28  0:10 [PATCHSET 0/3] fstests: regression tests for 5.13/5.14 rt fixes Darrick J. Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).