[PATCHSET 0/6] fstests: miscellaneous fixes

[PATCHSET 0/6] fstests: miscellaneous fixes

[Darrick J. Wong]

Hi all,

Various small fixes to the fstests suite, and some refactoring of common
idioms needed for testing realtime devices.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
 common/populate                                    |    2 -
 common/xfs                                         |   25 +++++++++++
 ltp/fsstress.c                                     |   24 +++++-----
 ltp/fsx.c                                          |   22 +++++----
 .../aio-dio-append-write-fallocate-race.c          |    2 -
 src/global.h                                       |   13 ++++++
 tests/generic/223                                  |    3 +
 tests/generic/449                                  |    2 -
 tests/xfs/004                                      |    2 -
 tests/xfs/088                                      |    1 
 tests/xfs/089                                      |    1 
 tests/xfs/091                                      |    1 
 tests/xfs/117                                      |   47 ++++++++++++--------
 tests/xfs/120                                      |    1 
 tests/xfs/130                                      |    1 
 tests/xfs/139                                      |    2 +
 tests/xfs/146                                      |    2 -
 tests/xfs/147                                      |    2 -
 tests/xfs/178                                      |    4 +-
 tests/xfs/207                                      |    1 
 tests/xfs/229                                      |    1 
 tests/xfs/235                                      |    1 
 tests/xfs/272                                      |    2 -
 tests/xfs/318                                      |    2 -
 tests/xfs/431                                      |    4 +-
 tests/xfs/521                                      |    2 -
 tests/xfs/528                                      |    2 -
 tests/xfs/532                                      |    2 -
 tests/xfs/533                                      |    2 -
 tests/xfs/538                                      |    2 -
 30 files changed, 119 insertions(+), 59 deletions(-)

[PATCH 1/6] common/xfs: refactor commands to select a particular xfs backing device

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Refactor all the places where we try to force new file data allocations
to a specific xfs backing device so that we don't end up open-coding the
same xfs_io command lines over and over.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/populate   |    2 +-
 common/xfs        |   25 +++++++++++++++++++++++++
 tests/generic/223 |    3 ++-
 tests/generic/449 |    2 +-
 tests/xfs/004     |    2 +-
 tests/xfs/146     |    2 +-
 tests/xfs/147     |    2 +-
 tests/xfs/272     |    2 +-
 tests/xfs/318     |    2 +-
 tests/xfs/431     |    4 ++--
 tests/xfs/521     |    2 +-
 tests/xfs/528     |    2 +-
 tests/xfs/532     |    2 +-
 tests/xfs/533     |    2 +-
 tests/xfs/538     |    2 +-
 15 files changed, 41 insertions(+), 15 deletions(-)


diff --git a/common/populate b/common/populate
index d484866a..867776cd 100644
--- a/common/populate
+++ b/common/populate
@@ -162,7 +162,7 @@ _scratch_xfs_populate() {
 	# Clear the rtinherit flag on the root directory so that files are
 	# always created on the data volume regardless of MKFS_OPTIONS.  We can
 	# set the realtime flag when needed.
-	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+	_xfs_force_bdev data $SCRATCH_MNT
 
 	blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
 	dblksz="$($XFS_INFO_PROG "${SCRATCH_MNT}" | grep naming.*bsize | sed -e 's/^.*bsize=//g' -e 's/\([0-9]*\).*$/\1/g')"
diff --git a/common/xfs b/common/xfs
index 725819bd..d7f2a005 100644
--- a/common/xfs
+++ b/common/xfs
@@ -194,6 +194,31 @@ _xfs_get_file_block_size()
 	$XFS_INFO_PROG "$path" | grep realtime | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g'
 }
 
+# Set or clear the realtime status of every supplied path.  The first argument
+# is either 'data' or 'realtime'.  All other arguments should be paths to
+# existing directories or empty regular files.
+#
+# For each directory, each file subsequently created will target the given
+# device for file data allocations.  For each empty regular file, each
+# subsequent file data allocation will be on the given device.
+_xfs_force_bdev()
+{
+	local device="$1"
+	shift
+	local chattr_arg=""
+
+	case "$device" in
+	"data")		chattr_arg="-t";;
+	"realtime")	chattr_arg="+t";;
+	*)
+		echo "${device}: Don't know what device this is?"
+		return 1
+		;;
+	esac
+
+	$XFS_IO_PROG -c "chattr $chattr_arg" "$@"
+}
+
 _xfs_get_fsxattr()
 {
 	local field="$1"
diff --git a/tests/generic/223 b/tests/generic/223
index f6393293..078fd725 100755
--- a/tests/generic/223
+++ b/tests/generic/223
@@ -46,7 +46,8 @@ for SUNIT_K in 8 16 32 64 128; do
 	# This test checks for stripe alignments of space allocations on the
 	# filesystem.  Make sure all files get created on the main device,
 	# which for XFS means no rt files.
-	test "$FSTYP" = "xfs" && $XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+	test "$FSTYP" = "xfs" && \
+		_xfs_force_bdev data $SCRATCH_MNT
 
 	for SIZE_MULT in 1 2 8 64 256; do
 		let SIZE=$SIZE_MULT*$SUNIT_BYTES
diff --git a/tests/generic/449 b/tests/generic/449
index 5fd15367..2a5065f6 100755
--- a/tests/generic/449
+++ b/tests/generic/449
@@ -46,7 +46,7 @@ _scratch_mount || _fail "mount failed"
 # This is a test of xattr behavior when we run out of disk space for xattrs,
 # so make sure the pwrite goes to the data device and not the rt volume.
 test "$FSTYP" = "xfs" && \
-	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+	_xfs_force_bdev data $SCRATCH_MNT
 
 TFILE=$SCRATCH_MNT/testfile.$seq
 
diff --git a/tests/xfs/004 b/tests/xfs/004
index 7633071c..d3fb9c95 100755
--- a/tests/xfs/004
+++ b/tests/xfs/004
@@ -31,7 +31,7 @@ _populate_scratch()
 	# This test looks at specific behaviors of the xfs_db freesp command,
 	# which reports on the contents of the free space btrees for the data
 	# device.  Don't let anything get created on the realtime volume.
-	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+	_xfs_force_bdev data $SCRATCH_MNT
 	dd if=/dev/zero of=$SCRATCH_MNT/foo count=200 bs=4096 >/dev/null 2>&1 &
 	dd if=/dev/zero of=$SCRATCH_MNT/goo count=400 bs=4096 >/dev/null 2>&1 &
 	dd if=/dev/zero of=$SCRATCH_MNT/moo count=800 bs=4096 >/dev/null 2>&1 &
diff --git a/tests/xfs/146 b/tests/xfs/146
index 8f85024d..49020e01 100755
--- a/tests/xfs/146
+++ b/tests/xfs/146
@@ -78,7 +78,7 @@ _scratch_mkfs -r size=$rtsize >> $seqres.full
 _scratch_mount >> $seqres.full
 
 # Make sure the root directory has rtinherit set so our test file will too
-$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+_xfs_force_bdev realtime $SCRATCH_MNT
 
 # Allocate some stuff at the start, to force the first falloc of the ouch file
 # to happen somewhere in the middle of the rt volume
diff --git a/tests/xfs/147 b/tests/xfs/147
index da962f96..5cc363aa 100755
--- a/tests/xfs/147
+++ b/tests/xfs/147
@@ -50,7 +50,7 @@ rextblks=$((rextsize / blksz))
 echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
 
 # Make sure the root directory has rtinherit set so our test file will too
-$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+_xfs_force_bdev realtime $SCRATCH_MNT
 
 sz=$((rextsize * 100))
 range="$((blksz * 3)) $blksz"
diff --git a/tests/xfs/272 b/tests/xfs/272
index 6c0fede5..6a1372b8 100755
--- a/tests/xfs/272
+++ b/tests/xfs/272
@@ -38,7 +38,7 @@ _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 
 # Make sure everything is on the data device
-$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+_xfs_force_bdev data $SCRATCH_MNT
 
 _pwrite_byte 0x80 0 737373 $SCRATCH_MNT/urk >> $seqres.full
 sync
diff --git a/tests/xfs/318 b/tests/xfs/318
index 07375b1f..07b4f7f1 100755
--- a/tests/xfs/318
+++ b/tests/xfs/318
@@ -44,7 +44,7 @@ _scratch_mount >> $seqres.full
 
 # This test depends on specific behaviors of the data device, so create all
 # files on it.
-$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "Create files"
 touch $SCRATCH_MNT/file1
diff --git a/tests/xfs/431 b/tests/xfs/431
index e67906dc..797b8fcd 100755
--- a/tests/xfs/431
+++ b/tests/xfs/431
@@ -47,7 +47,7 @@ _scratch_mount
 
 # Set realtime inherit flag on scratch mount, suppress output
 # as this may simply error out on future kernels
-$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT &> /dev/null
+_xfs_force_bdev realtime $SCRATCH_MNT &> /dev/null
 
 # Check if 't' is actually set, as xfs_io returns 0 even when it fails to set
 # an attribute. And erroring out here is fine, this would be desired behavior
@@ -60,7 +60,7 @@ if $XFS_IO_PROG -c 'lsattr' $SCRATCH_MNT | grep -q 't'; then
 	# Remove the testfile and rt inherit flag after we are done or
 	# xfs_repair will fail.
 	rm -f $SCRATCH_MNT/testfile
-	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT | tee -a $seqres.full 2>&1
+	_xfs_force_bdev data $SCRATCH_MNT | tee -a $seqres.full 2>&1
 fi
 
 # success, all done
diff --git a/tests/xfs/521 b/tests/xfs/521
index b8026d45..99408a06 100755
--- a/tests/xfs/521
+++ b/tests/xfs/521
@@ -55,7 +55,7 @@ testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
 
 echo "Check rt volume stats"
-$XFS_IO_PROG -c 'chattr +t' $testdir
+_xfs_force_bdev realtime $testdir
 $XFS_INFO_PROG $SCRATCH_MNT >> $seqres.full
 before=$(stat -f -c '%b' $testdir)
 
diff --git a/tests/xfs/528 b/tests/xfs/528
index 7f98c5b8..d483ae82 100755
--- a/tests/xfs/528
+++ b/tests/xfs/528
@@ -77,7 +77,7 @@ test_ops() {
 		_notrun "Could not mount rextsize=$rextsize with synthetic rt volume"
 
 	# Force all files to be realtime files
-	$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+	_xfs_force_bdev realtime $SCRATCH_MNT
 
 	log "Test regular write, rextsize=$rextsize"
 	mk_file $SCRATCH_MNT/write $rextsize
diff --git a/tests/xfs/532 b/tests/xfs/532
index 560af586..f421070a 100755
--- a/tests/xfs/532
+++ b/tests/xfs/532
@@ -47,7 +47,7 @@ _scratch_mount >> $seqres.full
 
 # Disable realtime inherit flag (if any) on root directory so that space on data
 # device gets fragmented rather than realtime device.
-$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+_xfs_force_bdev data $SCRATCH_MNT
 
 bsize=$(_get_block_size $SCRATCH_MNT)
 
diff --git a/tests/xfs/533 b/tests/xfs/533
index dd4cb4c4..1c8cc925 100755
--- a/tests/xfs/533
+++ b/tests/xfs/533
@@ -58,7 +58,7 @@ _scratch_mount >> $seqres.full
 
 # Disable realtime inherit flag (if any) on root directory so that space on data
 # device gets fragmented rather than realtime device.
-$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "Consume free space"
 fillerdir=$SCRATCH_MNT/fillerdir
diff --git a/tests/xfs/538 b/tests/xfs/538
index 97273b88..dc0e5f28 100755
--- a/tests/xfs/538
+++ b/tests/xfs/538
@@ -44,7 +44,7 @@ _scratch_mount >> $seqres.full
 
 # Disable realtime inherit flag (if any) on root directory so that space on data
 # device gets fragmented rather than realtime device.
-$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+_xfs_force_bdev data $SCRATCH_MNT
 
 bsize=$(_get_file_block_size $SCRATCH_MNT)
 

[PATCH 2/6] xfs: force file creation to the data device for certain layout tests

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

I found a bunch more tests in the xfs/ directory that try to create
specific metadata layouts on the data device, either because they're
fuzz tests or because they're testing specific edge cases of the code
base.  Either way, these test need to override '-d rtinherit' in the
MKFS_OPTIONS, so do that with _xfs_force_bdev.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/088 |    1 +
 tests/xfs/089 |    1 +
 tests/xfs/091 |    1 +
 tests/xfs/120 |    1 +
 tests/xfs/130 |    1 +
 tests/xfs/139 |    2 ++
 tests/xfs/207 |    1 +
 tests/xfs/229 |    1 +
 tests/xfs/235 |    1 +
 9 files changed, 10 insertions(+)


diff --git a/tests/xfs/088 b/tests/xfs/088
index 6c5cbec8..b3e65dcf 100755
--- a/tests/xfs/088
+++ b/tests/xfs/088
@@ -48,6 +48,7 @@ _scratch_mkfs_xfs > /dev/null
 echo "+ mount fs image"
 _scratch_mount
 blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "+ make some files"
 mkdir -p "${TESTDIR}"
diff --git a/tests/xfs/089 b/tests/xfs/089
index 2892ad9e..21380798 100755
--- a/tests/xfs/089
+++ b/tests/xfs/089
@@ -48,6 +48,7 @@ _scratch_mkfs_xfs > /dev/null
 echo "+ mount fs image"
 _scratch_mount
 blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "+ make some files"
 mkdir -p "${TESTDIR}"
diff --git a/tests/xfs/091 b/tests/xfs/091
index 04322cec..ff8f0f1f 100755
--- a/tests/xfs/091
+++ b/tests/xfs/091
@@ -48,6 +48,7 @@ _scratch_mkfs_xfs > /dev/null
 echo "+ mount fs image"
 _scratch_mount
 blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "+ make some files"
 mkdir -p "${TESTDIR}"
diff --git a/tests/xfs/120 b/tests/xfs/120
index e66bc763..f5eb14cc 100755
--- a/tests/xfs/120
+++ b/tests/xfs/120
@@ -47,6 +47,7 @@ echo "+ mount fs image"
 _scratch_mount
 blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
 nr="$((blksz * 2 / 16))"
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "+ make some files"
 $XFS_IO_PROG -f -c "pwrite -S 0x62 0 $((blksz * nr))" -c 'fsync' "${SCRATCH_MNT}/bigfile" >> $seqres.full
diff --git a/tests/xfs/130 b/tests/xfs/130
index 3071eace..6f6e8512 100755
--- a/tests/xfs/130
+++ b/tests/xfs/130
@@ -43,6 +43,7 @@ echo "+ mount fs image"
 _scratch_mount
 blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
 agcount="$(_xfs_mount_agcount $SCRATCH_MNT)"
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "+ make some files"
 _pwrite_byte 0x62 0 $((blksz * 64)) "${SCRATCH_MNT}/file0" >> "$seqres.full"
diff --git a/tests/xfs/139 b/tests/xfs/139
index 1444444d..58b71711 100755
--- a/tests/xfs/139
+++ b/tests/xfs/139
@@ -38,12 +38,14 @@ rm -f $seqres.full
 
 _scratch_mkfs >/dev/null 2>&1
 _scratch_mount
+_xfs_force_bdev data $SCRATCH_MNT
 blksz=$(_get_file_block_size $SCRATCH_MNT)
 _scratch_unmount
 
 echo "Format and mount"
 _scratch_mkfs -d agsize=$((16384 * $blksz)) > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
+_xfs_force_bdev data $SCRATCH_MNT
 
 testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
diff --git a/tests/xfs/207 b/tests/xfs/207
index f703c0dc..f0f30754 100755
--- a/tests/xfs/207
+++ b/tests/xfs/207
@@ -50,6 +50,7 @@ rm -f $seqres.full
 echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
+_xfs_force_bdev data $SCRATCH_MNT
 
 testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
diff --git a/tests/xfs/229 b/tests/xfs/229
index e723b10b..64851557 100755
--- a/tests/xfs/229
+++ b/tests/xfs/229
@@ -38,6 +38,7 @@ _require_fs_space $TEST_DIR 3200000
 TDIR="${TEST_DIR}/t_holes"
 NFILES="10"
 EXTSIZE="256k"
+_xfs_force_bdev data $TEST_DIR
 
 # Create the test directory
 mkdir ${TDIR}
diff --git a/tests/xfs/235 b/tests/xfs/235
index a2ab9e55..55f5c5a6 100755
--- a/tests/xfs/235
+++ b/tests/xfs/235
@@ -41,6 +41,7 @@ echo "+ mount fs image"
 _scratch_mount
 blksz=$(stat -f -c '%s' ${SCRATCH_MNT})
 agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
+_xfs_force_bdev data $SCRATCH_MNT
 
 echo "+ make some files"
 _pwrite_byte 0x62 0 $((blksz * 64)) ${SCRATCH_MNT}/file0 >> $seqres.full

[PATCH 3/6] xfs/117: fix fragility in this fuzz test

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This fuzz test has some fragility problems -- it doesn't do anything to
guarantee that the inodes that it checks for EFSCORRUPTED are the same
ones that it fuzzed, and it doesn't explicitly try to avoid victimizing
inodes in the same chunk as the root directory.  As a result, this test
fails annoyingly frequently.

Fix both of these problems and get rid of the confusingly named TESTDIR
variable.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/117 |   47 +++++++++++++++++++++++++++++------------------
 1 file changed, 29 insertions(+), 18 deletions(-)


diff --git a/tests/xfs/117 b/tests/xfs/117
index d3f4675f..32be525f 100755
--- a/tests/xfs/117
+++ b/tests/xfs/117
@@ -39,8 +39,7 @@ _require_xfs_db_blocktrash_z_command
 test -z "${FUZZ_ARGS}" && FUZZ_ARGS="-n 8 -3"
 
 rm -f $seqres.full
-TESTDIR="${SCRATCH_MNT}/scratchdir"
-TESTFILE="${TESTDIR}/testfile"
+victimdir="${SCRATCH_MNT}/scratchdir"
 
 echo "+ create scratch fs"
 _scratch_mkfs_xfs > /dev/null
@@ -50,37 +49,49 @@ _scratch_mount
 blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
 
 echo "+ make some files"
-mkdir -p "${TESTDIR}"
-for x in `seq 1 1024`; do
-	touch "${SCRATCH_MNT}/junk.${x}"
-	inode="$(stat -c '%i' "${SCRATCH_MNT}/junk.${x}")"
-	if [ "$x" -gt 512 ] && [ "$((inode % 64))" -eq 0 ]; then
-		mv "${SCRATCH_MNT}/junk.${x}" "${TESTFILE}.1"
-		break
-	fi
+mkdir -p "$victimdir"
+
+rootdir="$(stat -c '%i' "$SCRATCH_MNT")"
+rootchunk=$(( rootdir / 64 ))
+
+# First we create some dummy file so that the victim files don't get created
+# in the same inode chunk as the root directory, because a corrupt inode in
+# the root chunk causes mount to fail.
+for ((i = 0; i < 256; i++)); do
+	fname="$SCRATCH_MNT/dummy.$i"
+	touch "$fname"
+	ino="$(stat -c '%i' "$fname")"
+	ichunk=$(( ino / 64 ))
+	test "$ichunk" -gt "$rootchunk" && break
 done
-for x in `seq 2 64`; do
-	touch "${TESTFILE}.${x}"
+
+# Now create some victim files
+inos=()
+for ((i = 0; i < 64; i++)); do
+	fname="$victimdir/test.$i"
+	touch "$fname"
+	inos+=("$(stat -c '%i' "$fname")")
 done
-inode="$(stat -c '%i' "${TESTFILE}.1")"
+echo "First victim inode is: " >> $seqres.full
+stat -c '%i' "$fname" >> $seqres.full
 umount "${SCRATCH_MNT}"
 
 echo "+ check fs"
 _scratch_xfs_repair -n >> $seqres.full 2>&1 || _fail "xfs_repair should not fail"
 
 echo "+ corrupt image"
-seq "${inode}" "$((inode + 63))" | while read ino; do
+for ino in "${inos[@]}"; do
 	_scratch_xfs_db -x -c "inode ${ino}" -c "stack" -c "blocktrash -x 32 -y $((blksz * 8)) -z ${FUZZ_ARGS}" >> $seqres.full 2>&1
 done
 
 echo "+ mount image && modify files"
 broken=1
 if _try_scratch_mount >> $seqres.full 2>&1; then
-
-	for x in `seq 1 64`; do
-		stat "${TESTFILE}.${x}" >> $seqres.full 2>&1
+	for ((i = 0; i < 64; i++)); do
+		fname="$victimdir/test.$i"
+		stat "$fname" &>> $seqres.full
 		test $? -eq 0 && broken=0
-		touch "${TESTFILE}.${x}" >> $seqres.full 2>&1
+		touch "$fname" &>> $seqres.full
 		test $? -eq 0 && broken=0
 	done
 	umount "${SCRATCH_MNT}"

[PATCH 4/6] fsx/fsstress: round blocksize properly

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The block sizes reported by stat and DIOINFO aren't required to be
powers of two.  This can happen on an XFS filesystem with a realtime
extent size that isn't a power of two; on such filesystems, certain IO
calls will fail due to alignment issues.  Fix that by providing rounding
helpers that work for all sizes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 ltp/fsstress.c |   24 ++++++++++++------------
 ltp/fsx.c      |   22 +++++++++++-----------
 src/global.h   |   13 +++++++++++++
 3 files changed, 36 insertions(+), 23 deletions(-)


diff --git a/ltp/fsstress.c b/ltp/fsstress.c
index e7cd0eae..b4ddf5e2 100644
--- a/ltp/fsstress.c
+++ b/ltp/fsstress.c
@@ -2140,7 +2140,7 @@ do_aio_rw(int opno, long r, int flags)
 			" fallback to stat()\n",
 				procid, opno, f.path, st, errno);
 		diob.d_mem = diob.d_miniosz = stb.st_blksize;
-		diob.d_maxiosz = INT_MAX & ~(diob.d_miniosz - 1);
+		diob.d_maxiosz = rounddown_64(INT_MAX, diob.d_miniosz);
 	}
 	dio_env = getenv("XFS_DIO_MIN");
 	if (dio_env)
@@ -2608,7 +2608,7 @@ clonerange_f(
 
 	/* Calculate offsets */
 	len = (random() % FILELEN_MAX) + 1;
-	len &= ~(stat1.st_blksize - 1);
+	len = rounddown_64(len, stat1.st_blksize);
 	if (len == 0)
 		len = stat1.st_blksize;
 	if (len > stat1.st_size)
@@ -2620,7 +2620,7 @@ clonerange_f(
 	else
 		off1 = (off64_t)(lr % MIN(stat1.st_size - len, MAXFSIZE));
 	off1 %= maxfsize;
-	off1 &= ~(stat1.st_blksize - 1);
+	off1 = rounddown_64(off1, stat1.st_blksize);
 
 	/*
 	 * If srcfile == destfile, randomly generate destination ranges
@@ -2631,7 +2631,7 @@ clonerange_f(
 		lr = ((int64_t)random() << 32) + random();
 		off2 = (off64_t)(lr % max_off2);
 		off2 %= maxfsize;
-		off2 &= ~(stat2.st_blksize - 1);
+		off2 = rounddown_64(off2, stat2.st_blksize);
 	} while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len);
 
 	/* Clone data blocks */
@@ -2968,7 +2968,7 @@ deduperange_f(
 
 	/* Never try to dedupe more than half of the src file. */
 	len = (random() % FILELEN_MAX) + 1;
-	len &= ~(stat[0].st_blksize - 1);
+	len = rounddown_64(len, stat[0].st_blksize);
 	if (len == 0)
 		len = stat[0].st_blksize / 2;
 	if (len > stat[0].st_size / 2)
@@ -2981,7 +2981,7 @@ deduperange_f(
 	else
 		off[0] = (off64_t)(lr % MIN(stat[0].st_size - len, MAXFSIZE));
 	off[0] %= maxfsize;
-	off[0] &= ~(stat[0].st_blksize - 1);
+	off[0] = rounddown_64(off[0], stat[0].st_blksize);
 
 	/*
 	 * If srcfile == destfile[i], randomly generate destination ranges
@@ -2997,7 +2997,7 @@ deduperange_f(
 			else
 				off[i] = (off64_t)(lr % MIN(stat[i].st_size - len, MAXFSIZE));
 			off[i] %= maxfsize;
-			off[i] &= ~(stat[i].st_blksize - 1);
+			off[i] = rounddown_64(off[i], stat[i].st_blksize);
 		} while (stat[0].st_ino == stat[i].st_ino &&
 			 llabs(off[i] - off[0]) < len &&
 			 tries++ < 10);
@@ -3406,7 +3406,7 @@ dread_f(int opno, long r)
 			" fallback to stat()\n",
 				procid, opno, f.path, st, errno);
 		diob.d_mem = diob.d_miniosz = stb.st_blksize;
-		diob.d_maxiosz = INT_MAX & ~(diob.d_miniosz - 1);
+		diob.d_maxiosz = rounddown_64(INT_MAX, diob.d_miniosz);
 	}
 
 	dio_env = getenv("XFS_DIO_MIN");
@@ -3483,7 +3483,7 @@ dwrite_f(int opno, long r)
 				" %s%s return %d, fallback to stat()\n",
 			       procid, opno, f.path, st, errno);
 		diob.d_mem = diob.d_miniosz = stb.st_blksize;
-		diob.d_maxiosz = INT_MAX & ~(diob.d_miniosz - 1);
+		diob.d_maxiosz = rounddown_64(INT_MAX, diob.d_miniosz);
 	}
 
 	dio_env = getenv("XFS_DIO_MIN");
@@ -3579,8 +3579,8 @@ do_fallocate(int opno, long r, int mode)
 	 */
 	if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) &&
 		(opno % 2)) {
-		off = ((off + stb.st_blksize - 1) & ~(stb.st_blksize - 1));
-		len = ((len + stb.st_blksize - 1) & ~(stb.st_blksize - 1));
+		off = roundup_64(off, stb.st_blksize);
+		len = roundup_64(len, stb.st_blksize);
 	}
 	mode |= FALLOC_FL_KEEP_SIZE & random();
 	e = fallocate(fd, mode, (loff_t)off, (loff_t)len) < 0 ? errno : 0;
@@ -4186,7 +4186,7 @@ do_mmap(int opno, long r, int prot)
 
 	lr = ((int64_t)random() << 32) + random();
 	off = (off64_t)(lr % stb.st_size);
-	off &= (off64_t)(~(sysconf(_SC_PAGE_SIZE) - 1));
+	off = rounddown_64(off, sysconf(_SC_PAGE_SIZE));
 	len = (size_t)(random() % MIN(stb.st_size - off, FILELEN_MAX)) + 1;
 
 	flags = (random() % 2) ? MAP_SHARED : MAP_PRIVATE;
diff --git a/ltp/fsx.c b/ltp/fsx.c
index d526d294..12c2cc33 100644
--- a/ltp/fsx.c
+++ b/ltp/fsx.c
@@ -203,7 +203,7 @@ static void *round_ptr_up(void *ptr, unsigned long align, unsigned long offset)
 {
 	unsigned long ret = (unsigned long)ptr;
 
-	ret = ((ret + align - 1) & ~(align - 1));
+	ret = roundup_64(ret, align);
 	ret += offset;
 	return (void *)ret;
 }
@@ -1948,12 +1948,12 @@ static void generate_dest_range(bool bdy_align,
 
 	TRIM_OFF_LEN(*src_offset, *size, file_size);
 	if (bdy_align) {
-		*src_offset -= *src_offset % readbdy;
+		*src_offset = rounddown_64(*src_offset, readbdy);
 		if (o_direct)
-			*size -= *size % readbdy;
+			*size = rounddown_64(*size, readbdy);
 	} else {
-		*src_offset = *src_offset & ~(block_size - 1);
-		*size = *size & ~(block_size - 1);
+		*src_offset = rounddown_64(*src_offset, block_size);
+		*size = rounddown_64(*size, block_size);
 	}
 
 	do {
@@ -1964,9 +1964,9 @@ static void generate_dest_range(bool bdy_align,
 		*dst_offset = random();
 		TRIM_OFF(*dst_offset, max_range_end);
 		if (bdy_align)
-			*dst_offset -= *dst_offset % writebdy;
+			*dst_offset = rounddown_64(*dst_offset, writebdy);
 		else
-			*dst_offset = *dst_offset & ~(block_size - 1);
+			*dst_offset = rounddown_64(*dst_offset, block_size);
 	} while (range_overlaps(*src_offset, *dst_offset, *size) ||
 		 *dst_offset + *size > max_range_end);
 }
@@ -2156,8 +2156,8 @@ test(void)
 		break;
 	case OP_COLLAPSE_RANGE:
 		TRIM_OFF_LEN(offset, size, file_size - 1);
-		offset = offset & ~(block_size - 1);
-		size = size & ~(block_size - 1);
+		offset = rounddown_64(offset, block_size);
+		size = rounddown_64(size, block_size);
 		if (size == 0) {
 			log4(OP_COLLAPSE_RANGE, offset, size, FL_SKIPPED);
 			goto out;
@@ -2167,8 +2167,8 @@ test(void)
 	case OP_INSERT_RANGE:
 		TRIM_OFF(offset, file_size);
 		TRIM_LEN(file_size, size, maxfilelen);
-		offset = offset & ~(block_size - 1);
-		size = size & ~(block_size - 1);
+		offset = rounddown_64(offset, block_size);
+		size = rounddown_64(size, block_size);
 		if (size == 0) {
 			log4(OP_INSERT_RANGE, offset, size, FL_SKIPPED);
 			goto out;
diff --git a/src/global.h b/src/global.h
index e5e46234..b4407099 100644
--- a/src/global.h
+++ b/src/global.h
@@ -171,4 +171,17 @@
 #include <sys/mman.h>
 #endif
 
+static inline unsigned long long
+rounddown_64(unsigned long long x, unsigned int y)
+{
+	x /= y;
+	return x * y;
+}
+
+static inline unsigned long long
+roundup_64(unsigned long long x, unsigned int y)
+{
+	return rounddown_64(x + y - 1, y);
+}
+
 #endif /* GLOBAL_H */

[PATCH 5/6] xfs/178: fix mkfs success test

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix the obviously incorrect code here that wants to fail the test if
mkfs doesn't succeed.  The return value ("$?") is always the status of
the /last/ command in the pipe.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/178 |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/178 b/tests/xfs/178
index a24ef50c..122d63d2 100755
--- a/tests/xfs/178
+++ b/tests/xfs/178
@@ -57,8 +57,8 @@ _supported_fs xfs
 #             fix filesystem, new mkfs.xfs will be fine.
 
 _require_scratch
-_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs \
-        || _fail "mkfs failed!"
+_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
+test "${PIPESTATUS[0]}" -eq 0 || _fail "mkfs failed!"
 
 # By executing the followint tmp file, will get on the mkfs options stored in
 # variables

[PATCH 6/6] aio-dio-append-write-fallocate-race: fix directio buffer alignment bugs

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This program fails on filesystems where the stat() block size isn't a
strict power of two because it foolishly feeds that to posix_memalign to
allocate an aligned memory buffer for directio.  posix_memalign requires
the alignment value to be a power of two, so generic/586 fails.

The system page size generally works well for directio buffers, so use
that instead.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 .../aio-dio-append-write-fallocate-race.c          |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c b/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c
index 091b047d..d3a2e5fc 100644
--- a/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c
+++ b/src/aio-dio-regress/aio-dio-append-write-fallocate-race.c
@@ -65,7 +65,7 @@ test(
 	}
 	blocksize = sbuf.st_blksize;
 
-	ret = posix_memalign((void **)&buf, blocksize, blocksize);
+	ret = posix_memalign((void **)&buf, sysconf(_SC_PAGESIZE), blocksize);
 	if (ret) {
 		errno = ret;
 		perror("buffer");

Re: [PATCH 1/6] common/xfs: refactor commands to select a particular xfs backing device

[Christoph Hellwig]

On Wed, May 19, 2021 at 04:56:46PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Refactor all the places where we try to force new file data allocations
> to a specific xfs backing device so that we don't end up open-coding the
> same xfs_io command lines over and over.

Eventually this might need generalization for other file systems for the
generic tests, but for now this looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 2/6] xfs: force file creation to the data device for certain layout tests

[Christoph Hellwig]

On Wed, May 19, 2021 at 04:56:52PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> I found a bunch more tests in the xfs/ directory that try to create
> specific metadata layouts on the data device, either because they're
> fuzz tests or because they're testing specific edge cases of the code
> base.  Either way, these test need to override '-d rtinherit' in the
> MKFS_OPTIONS, so do that with _xfs_force_bdev.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 3/6] xfs/117: fix fragility in this fuzz test

[Christoph Hellwig]

On Wed, May 19, 2021 at 04:56:58PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This fuzz test has some fragility problems -- it doesn't do anything to
> guarantee that the inodes that it checks for EFSCORRUPTED are the same
> ones that it fuzzed, and it doesn't explicitly try to avoid victimizing
> inodes in the same chunk as the root directory.  As a result, this test
> fails annoyingly frequently.
> 
> Fix both of these problems and get rid of the confusingly named TESTDIR
> variable.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 4/6] fsx/fsstress: round blocksize properly

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 5/6] xfs/178: fix mkfs success test

[Christoph Hellwig]

On Wed, May 19, 2021 at 04:57:10PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Fix the obviously incorrect code here that wants to fail the test if
> mkfs doesn't succeed.  The return value ("$?") is always the status of
> the /last/ command in the pipe.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 6/6] aio-dio-append-write-fallocate-race: fix directio buffer alignment bugs

[Christoph Hellwig]

On Wed, May 19, 2021 at 04:57:16PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This program fails on filesystems where the stat() block size isn't a
> strict power of two because it foolishly feeds that to posix_memalign to
> allocate an aligned memory buffer for directio.  posix_memalign requires
> the alignment value to be a power of two, so generic/586 fails.
> 
> The system page size generally works well for directio buffers, so use
> that instead.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>