[PATCHSET 0/9] fstests: random fixes

[PATCHSET 0/9] fstests: random fixes

[Darrick J. Wong]

Hi all,

Here's the usual batch of odd fixes for fstests.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
 check                  |    3 +++
 common/repair          |    1 +
 src/seek_sanity_test.c |   12 ++++++++++-
 tests/xfs/018          |   52 +++++++++++++++++++++++++++++++++++++++++++-----
 tests/xfs/018.out      |   16 ++++-----------
 tests/xfs/109          |    2 +-
 tests/xfs/166          |   19 ++++++++++++++----
 tests/xfs/547          |   14 +++++++++----
 tests/xfs/843          |   51 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/843.out      |    2 ++
 tests/xfs/844          |   33 ++++++++++++++++++++++++++++++
 tests/xfs/844.out      |    3 +++
 12 files changed, 181 insertions(+), 27 deletions(-)
 create mode 100755 tests/xfs/843
 create mode 100644 tests/xfs/843.out
 create mode 100755 tests/xfs/844
 create mode 100644 tests/xfs/844.out

[PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The seek sanity test tries to figure out a file space allocation unit by
calling stat and then using an iterative SEEK_DATA method to try to
detect a smaller blocksize based on SEEK_DATA's consultation of the
filesystem's internal block mapping.  This was put in (AFAICT) because
XFS' stat implementation returns max(filesystem blocksize, PAGESIZE) for
most regular files.

Unfortunately, for a realtime file with an extent size larger than a
single filesystem block this doesn't work at all because block mappings
still work at filesystem block granularity, but allocation units do not.
To fix this, detect the specific case where st_blksize != PAGE_SIZE and
trust the fstat results.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 src/seek_sanity_test.c |   12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)


diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
index 76587b7f..1030d0c5 100644
--- a/src/seek_sanity_test.c
+++ b/src/seek_sanity_test.c
@@ -45,6 +45,7 @@ static int get_io_sizes(int fd)
 	off_t pos = 0, offset = 1;
 	struct stat buf;
 	int shift, ret;
+	int pagesz = sysconf(_SC_PAGE_SIZE);
 
 	ret = fstat(fd, &buf);
 	if (ret) {
@@ -53,8 +54,16 @@ static int get_io_sizes(int fd)
 		return ret;
 	}
 
-	/* st_blksize is typically also the allocation size */
+	/*
+	 * st_blksize is typically also the allocation size.  However, XFS
+	 * rounds this up to the page size, so if the stat blocksize is exactly
+	 * one page, use this iterative algorithm to see if SEEK_DATA will hint
+	 * at a more precise answer based on the filesystem's (pre)allocation
+	 * decisions.
+	 */
 	alloc_size = buf.st_blksize;
+	if (alloc_size != pagesz)
+		goto done;
 
 	/* try to discover the actual alloc size */
 	while (pos == 0 && offset < alloc_size) {
@@ -80,6 +89,7 @@ static int get_io_sizes(int fd)
 	if (!shift)
 		offset += pos ? 0 : 1;
 	alloc_size = offset;
+done:
 	fprintf(stdout, "Allocation size: %ld\n", alloc_size);
 	return 0;
 

[PATCH 2/9] xfs/070: filter new superblock verifier messages

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In Linux 5.19, the superblock verifier logging changed to elaborate on
what was wrong.  Fix the xfs_repair filtering function to accomodate
this development.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/repair |    1 +
 1 file changed, 1 insertion(+)


diff --git a/common/repair b/common/repair
index 463ef9db..398e9904 100644
--- a/common/repair
+++ b/common/repair
@@ -29,6 +29,7 @@ _filter_repair()
 # for sb
 /- agno = / && next;	# remove each AG line (variable number)
 s/(pointer to) (\d+)/\1 INO/;
+s/Superblock has bad magic number.*/bad magic number/;
 # Changed inode output in 5.5.0
 s/sb root inode value /sb root inode /;
 s/realtime bitmap inode value /realtime bitmap inode /;

[PATCH 3/9] xfs: test mkfs.xfs sizing of internal logs that

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test that exercises the mkfs.xfs code that creates
log sizes that are very close to the AG size when stripe units are in
play and/or when the log is forced to be in AG 0.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/843     |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/843.out |    2 ++
 2 files changed, 53 insertions(+)
 create mode 100755 tests/xfs/843
 create mode 100644 tests/xfs/843.out


diff --git a/tests/xfs/843 b/tests/xfs/843
new file mode 100755
index 00000000..5bb4bfb4
--- /dev/null
+++ b/tests/xfs/843
@@ -0,0 +1,51 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test 843
+#
+# Now that we've increased the default log size calculation, test mkfs with
+# various stripe units and filesystem sizes to see if we can provoke mkfs into
+# breaking.
+#
+. ./common/preamble
+_begin_fstest auto mkfs
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs xfs
+_require_test
+echo Silence is golden
+
+testfile=$TEST_DIR/a
+rm -f $testfile
+
+test_format() {
+	local tag="$1"
+	shift
+
+	echo "$tag" >> $seqres.full
+	$MKFS_XFS_PROG $@ -d file,name=$testfile &>> $seqres.full
+	local res=$?
+	test $res -eq 0 || echo "$tag FAIL $res" | tee -a $seqres.full
+}
+
+# First we try various small filesystems and stripe sizes.
+for M in `seq 298 302` `seq 490 520`; do
+	for S in `seq 32 4 64`; do
+		test_format "M=$M S=$S" -dsu=${S}k,sw=1,size=${M}m -N
+	done
+done
+
+# Log so large it pushes the root dir into AG 1.  We can't use -N for the mkfs
+# because this check only occurs after the root directory has been allocated,
+# which mkfs -N doesn't do.
+test_format "log pushes rootdir into AG 1" -d agcount=3200,size=6366g -lagnum=0 -N
+
+# log end rounded beyond EOAG due to stripe unit
+test_format "log end beyond eoag" -d agcount=3200,size=6366g -d su=256k,sw=4 -N
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/843.out b/tests/xfs/843.out
new file mode 100644
index 00000000..87c13504
--- /dev/null
+++ b/tests/xfs/843.out
@@ -0,0 +1,2 @@
+QA output created by 843
+Silence is golden

[PATCH 4/9] xfs: test xfs_copy doesn't do cached read before libxfs_mount

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test for an xfs_copy fix that ensures that it
doesn't perform a cached read of an XFS filesystem prior to initializing
libxfs, since the xfs_mount (and hence the buffer cache) isn't set up
yet.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/844     |   33 +++++++++++++++++++++++++++++++++
 tests/xfs/844.out |    3 +++
 2 files changed, 36 insertions(+)
 create mode 100755 tests/xfs/844
 create mode 100644 tests/xfs/844.out


diff --git a/tests/xfs/844 b/tests/xfs/844
new file mode 100755
index 00000000..688abe33
--- /dev/null
+++ b/tests/xfs/844
@@ -0,0 +1,33 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test 844
+#
+# Regression test for xfsprogs commit:
+#
+# XXXXXXXX ("xfs_copy: don't use cached buffer reads until after libxfs_mount")
+#
+. ./common/preamble
+_begin_fstest auto copy
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_require_xfs_copy
+_require_test
+
+rm -f $TEST_DIR/$seq.*
+$XFS_IO_PROG -f -c 'truncate 100m' $TEST_DIR/$seq.a
+$XFS_IO_PROG -f -c 'truncate 100m' $TEST_DIR/$seq.b
+
+filter_copy() {
+	sed -e 's/Superblock has bad magic number.*/bad magic number/'
+}
+
+$XFS_COPY_PROG $TEST_DIR/$seq.a $TEST_DIR/$seq.b 2>&1 | filter_copy
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/844.out b/tests/xfs/844.out
new file mode 100644
index 00000000..dbefde1c
--- /dev/null
+++ b/tests/xfs/844.out
@@ -0,0 +1,3 @@
+QA output created by 844
+bad magic number
+xfs_copy: couldn't read superblock, error=22

[PATCH 5/9] check: document mkfs.xfs reliance on fstests exports

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

There are a number of fstests that employ special (and now unsupported)
XFS filesystem configurations to perform testing in a controlled
environment.  The presence of the QA_CHECK_FS and MSGVERB variables are
used by mkfs.xfs to detect that it's running inside fstests, which
enables the unsupported configurations.  Nobody else should be using
filesystems with tiny logs, non-redundant superblocks, or smaller than
the (new) minimum supported size.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 check |    3 +++
 1 file changed, 3 insertions(+)


diff --git a/check b/check
index 2ea2920f..4b0ebad6 100755
--- a/check
+++ b/check
@@ -33,6 +33,9 @@ _err_msg=""
 # start the initialisation work now
 iam=check
 
+# mkfs.xfs uses the presence of both of these variables to enable formerly
+# supported tiny filesystem configurations that fstests use for fuzz testing
+# in a controlled environment
 export MSGVERB="text:action"
 export QA_CHECK_FS=${QA_CHECK_FS:=true}
 

[PATCH 6/9] xfs/109: handle larger minimum filesystem size

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

mkfs will soon refuse to format a filesystem smaller than 300MB, so
increase the size of the filesystem to keep this test scenario
realistic.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/109 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/xfs/109 b/tests/xfs/109
index 6cb6917a..e3e491f1 100755
--- a/tests/xfs/109
+++ b/tests/xfs/109
@@ -78,7 +78,7 @@ if [ -n "$FASTSTART" -a -f $SCRATCH_MNT/f0 ]; then
 fi
 _scratch_unmount
 
-_scratch_mkfs_xfs -dsize=160m,agcount=4 $faststart | _filter_mkfs 2>$tmp.mkfs
+_scratch_mkfs_xfs -dsize=320m,agcount=4 $faststart | _filter_mkfs 2>$tmp.mkfs
 cat $tmp.mkfs >>$seqres.full
 _scratch_mount
 

[PATCH 7/9] xfs/018: fix LARP testing for small block sizes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix this test to work properly when the filesystem block size is less
than 4k.  Tripping the error injection points on shape changes in the
xattr structure must be done dynamically.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/018     |   52 +++++++++++++++++++++++++++++++++++++++++++++++-----
 tests/xfs/018.out |   16 ++++------------
 2 files changed, 51 insertions(+), 17 deletions(-)


diff --git a/tests/xfs/018 b/tests/xfs/018
index 041a3b24..14a6f716 100755
--- a/tests/xfs/018
+++ b/tests/xfs/018
@@ -54,6 +54,45 @@ test_attr_replay()
 	echo ""
 }
 
+test_attr_replay_loop()
+{
+	testfile=$testdir/$1
+	attr_name=$2
+	attr_value=$3
+	flag=$4
+	error_tag=$5
+
+	# Inject error
+	_scratch_inject_error $error_tag
+
+	# Set attribute; hopefully 1000 of them is enough to cause whatever
+	# attr structure shape change that the caller wants to test.
+	for ((i = 0; i < 1024; i++)); do
+		echo "$attr_value" | \
+			${ATTR_PROG} -$flag "$attr_name$i" $testfile > $tmp.out 2> $tmp.err
+		cat $tmp.out $tmp.err >> $seqres.full
+		cat $tmp.err | _filter_scratch | sed -e 's/attr_name[0-9]*/attr_nameXXXX/g'
+		touch $testfile &>/dev/null || break
+	done
+
+	# FS should be shut down, touch will fail
+	touch $testfile 2>&1 | _filter_scratch
+
+	# Remount to replay log
+	_scratch_remount_dump_log >> $seqres.full
+
+	# FS should be online, touch should succeed
+	touch $testfile
+
+	# Verify attr recovery
+	$ATTR_PROG -l $testfile >> $seqres.full
+	echo "Checking contents of $attr_name$i" >> $seqres.full
+	echo -n "${attr_name}XXXX: "
+	$ATTR_PROG -q -g $attr_name$i $testfile 2> /dev/null | md5sum;
+
+	echo ""
+}
+
 create_test_file()
 {
 	filename=$testdir/$1
@@ -88,6 +127,7 @@ echo 1 > /sys/fs/xfs/debug/larp
 attr16="0123456789ABCDEF"
 attr64="$attr16$attr16$attr16$attr16"
 attr256="$attr64$attr64$attr64$attr64"
+attr512="$attr256$attr256"
 attr1k="$attr256$attr256$attr256$attr256"
 attr4k="$attr1k$attr1k$attr1k$attr1k"
 attr8k="$attr4k$attr4k"
@@ -140,12 +180,14 @@ test_attr_replay extent_file1 "attr_name2" $attr1k "s" "larp"
 test_attr_replay extent_file1 "attr_name2" $attr1k "r" "larp"
 
 # extent, inject error on split
-create_test_file extent_file2 3 $attr1k
-test_attr_replay extent_file2 "attr_name4" $attr1k "s" "da_leaf_split"
+create_test_file extent_file2 0 $attr1k
+test_attr_replay_loop extent_file2 "attr_name" $attr1k "s" "da_leaf_split"
 
-# extent, inject error on fork transition
-create_test_file extent_file3 3 $attr1k
-test_attr_replay extent_file3 "attr_name4" $attr1k "s" "attr_leaf_to_node"
+# extent, inject error on fork transition.  The attr value must be less than
+# a full filesystem block so that the attrs don't use remote xattr values,
+# which means we miss the leaf to node transition.
+create_test_file extent_file3 0 $attr1k
+test_attr_replay_loop extent_file3 "attr_name" $attr512 "s" "attr_leaf_to_node"
 
 # extent, remote
 create_test_file extent_file4 1 $attr1k
diff --git a/tests/xfs/018.out b/tests/xfs/018.out
index 022b0ca3..c3021ee3 100644
--- a/tests/xfs/018.out
+++ b/tests/xfs/018.out
@@ -87,22 +87,14 @@ Attribute "attr_name1" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file
 attr_name2: d41d8cd98f00b204e9800998ecf8427e  -
 
 attr_set: Input/output error
-Could not set "attr_name4" for SCRATCH_MNT/testdir/extent_file2
+Could not set "attr_nameXXXX" for SCRATCH_MNT/testdir/extent_file2
 touch: cannot touch 'SCRATCH_MNT/testdir/extent_file2': Input/output error
-Attribute "attr_name4" has a 1025 byte value for SCRATCH_MNT/testdir/extent_file2
-Attribute "attr_name2" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file2
-Attribute "attr_name3" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file2
-Attribute "attr_name1" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file2
-attr_name4: 9fd415c49d67afc4b78fad4055a3a376  -
+attr_nameXXXX: 9fd415c49d67afc4b78fad4055a3a376  -
 
 attr_set: Input/output error
-Could not set "attr_name4" for SCRATCH_MNT/testdir/extent_file3
+Could not set "attr_nameXXXX" for SCRATCH_MNT/testdir/extent_file3
 touch: cannot touch 'SCRATCH_MNT/testdir/extent_file3': Input/output error
-Attribute "attr_name4" has a 1025 byte value for SCRATCH_MNT/testdir/extent_file3
-Attribute "attr_name2" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file3
-Attribute "attr_name3" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file3
-Attribute "attr_name1" has a 1024 byte value for SCRATCH_MNT/testdir/extent_file3
-attr_name4: 9fd415c49d67afc4b78fad4055a3a376  -
+attr_nameXXXX: a597dc41e4574873516420a7e4e5a3e0  -
 
 attr_set: Input/output error
 Could not set "attr_name2" for SCRATCH_MNT/testdir/extent_file4

[PATCH 8/9] xfs/166: fix golden output failures when multipage folios enabled

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Beginning with 5.18, some filesystems support creating large folios for
the page cache.  A system with 64k pages can create 256k folios, which
means that with the old file size of 1M, the last half of the file is
completely converted from unwritten to written by page_mkwrite.  The
test encodes a translated version of the xfs_bmap output in the golden
output, which means that the test now fails on 64k pages.  Fixing the
64k page case by increasing the file size to 2MB broke fsdax because
fsdax uses 2MB PMDs, hence 12MB.

Increase the size to prevent this from happening.  This may require
further revision if folios get larger or fsdax starts supporting PMDs
that are larger than 2MB.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/166 |   19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)


diff --git a/tests/xfs/166 b/tests/xfs/166
index 42379961..d45dc5e8 100755
--- a/tests/xfs/166
+++ b/tests/xfs/166
@@ -16,12 +16,12 @@ _begin_fstest rw metadata auto quick
 # the others are unwritten.
 _filter_blocks()
 {
-	$AWK_PROG '
+	$AWK_PROG -v file_size=$FILE_SIZE '
 /^ +[0-9]/ {
 	if (!written_size) {
 		written_size = $6
-		unwritten1 = ((1048576/512) / 2) - written_size
-		unwritten2 = ((1048576/512) / 2) - 2 * written_size
+		unwritten1 = ((file_size/512) / 2) - written_size
+		unwritten2 = ((file_size/512) / 2) - 2 * written_size
 	}
 
 	# is the extent unwritten?
@@ -58,7 +58,18 @@ _scratch_mount
 
 TEST_FILE=$SCRATCH_MNT/test_file
 TEST_PROG=$here/src/unwritten_mmap
-FILE_SIZE=1048576
+
+# Beginning with 5.18, some filesystems support creating large folios for the
+# page cache.  A system with 64k pages can create 256k folios, which means
+# that with the old file size of 1M, the last half of the file is completely
+# converted from unwritten to written by page_mkwrite.  The test will fail on
+# the golden output when this happens, so increase the size from the original
+# 1MB file size to at least (6 * 256k == 1.5MB) prevent this from happening.
+#
+# However, increasing the file size to around 2MB causes regressions when fsdax
+# is enabled because fsdax will try to use PMD entries for the mappings.  Hence
+# we need to set the file size to (6 * 2MB == 12MB) to cover all cases.
+FILE_SIZE=$((12 * 1048576))
 
 rm -f $TEST_FILE
 $TEST_PROG $FILE_SIZE $TEST_FILE

[PATCH 9/9] xfs/547: fix problems with realtime

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test needs to fragment the free space on the data device so that
each block added to the attr fork gets its own mapping.  If the test
configuration sets up a rt device and rtinherit=1 on the root dir, the
test will erroneously fragment space on the *realtime* volume.  When
this happens, attr fork allocations are contiguous and get merged into
fewer than 10 extents and the test fails.

Fix this test to force all allocations to be on the data device, and fix
incorrect variable usage in the error messages.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/547 |   14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)


diff --git a/tests/xfs/547 b/tests/xfs/547
index 9d4216ca..60121eb9 100755
--- a/tests/xfs/547
+++ b/tests/xfs/547
@@ -33,6 +33,10 @@ for nrext64 in 0 1; do
 		      >> $seqres.full
 	_scratch_mount >> $seqres.full
 
+	# Force data device extents so that we can fragment the free space
+	# and force attr fork allocations to be non-contiguous
+	_xfs_force_bdev data $SCRATCH_MNT
+
 	bsize=$(_get_file_block_size $SCRATCH_MNT)
 
 	testfile=$SCRATCH_MNT/testfile
@@ -76,13 +80,15 @@ for nrext64 in 0 1; do
 	acnt=$(_scratch_xfs_get_metadata_field core.naextents \
 					       "path /$(basename $testfile)")
 
-	if (( $dcnt != 10 )); then
-		echo "Invalid data fork extent count: $dextcnt"
+	echo "nrext64: $nrext64 dcnt: $dcnt acnt: $acnt" >> $seqres.full
+
+	if [ -z "$dcnt" ] || (( $dcnt != 10 )); then
+		echo "Invalid data fork extent count: $dcnt"
 		exit 1
 	fi
 
-	if (( $acnt < 10 )); then
-		echo "Invalid attr fork extent count: $aextcnt"
+	if [ -z "$acnt" ] || (( $acnt < 10 )); then
+		echo "Invalid attr fork extent count: $acnt"
 		exit 1
 	fi
 done

Re: [PATCH 2/9] xfs/070: filter new superblock verifier messages

[Dave Chinner]

On Tue, Jun 28, 2022 at 01:21:28PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> In Linux 5.19, the superblock verifier logging changed to elaborate on
> what was wrong.  Fix the xfs_repair filtering function to accomodate
> this development.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/repair |    1 +
>  1 file changed, 1 insertion(+)
> 
> 
> diff --git a/common/repair b/common/repair
> index 463ef9db..398e9904 100644
> --- a/common/repair
> +++ b/common/repair
> @@ -29,6 +29,7 @@ _filter_repair()
>  # for sb
>  /- agno = / && next;	# remove each AG line (variable number)
>  s/(pointer to) (\d+)/\1 INO/;
> +s/Superblock has bad magic number.*/bad magic number/;
>  # Changed inode output in 5.5.0
>  s/sb root inode value /sb root inode /;
>  s/realtime bitmap inode value /realtime bitmap inode /;

Didn't I already fix that in commit 4c76d0ba ("xfs/070: filter the
bad sb magic number error")?

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 3/9] xfs: test mkfs.xfs sizing of internal logs that

[Dave Chinner]

On Tue, Jun 28, 2022 at 01:21:34PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This is a regression test that exercises the mkfs.xfs code that creates
> log sizes that are very close to the AG size when stripe units are in
> play and/or when the log is forced to be in AG 0.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/843     |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/843.out |    2 ++
>  2 files changed, 53 insertions(+)
>  create mode 100755 tests/xfs/843
>  create mode 100644 tests/xfs/843.out
> 
> 
> diff --git a/tests/xfs/843 b/tests/xfs/843
> new file mode 100755
> index 00000000..5bb4bfb4
> --- /dev/null
> +++ b/tests/xfs/843
> @@ -0,0 +1,51 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test 843
> +#
> +# Now that we've increased the default log size calculation, test mkfs with
> +# various stripe units and filesystem sizes to see if we can provoke mkfs into
> +# breaking.
> +#
> +. ./common/preamble
> +_begin_fstest auto mkfs
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs xfs
> +_require_test
> +echo Silence is golden
> +
> +testfile=$TEST_DIR/a
> +rm -f $testfile
> +
> +test_format() {
> +	local tag="$1"
> +	shift
> +
> +	echo "$tag" >> $seqres.full
> +	$MKFS_XFS_PROG $@ -d file,name=$testfile &>> $seqres.full
> +	local res=$?
> +	test $res -eq 0 || echo "$tag FAIL $res" | tee -a $seqres.full
> +}
> +
> +# First we try various small filesystems and stripe sizes.
> +for M in `seq 298 302` `seq 490 520`; do
> +	for S in `seq 32 4 64`; do
> +		test_format "M=$M S=$S" -dsu=${S}k,sw=1,size=${M}m -N
> +	done
> +done
> +
> +# Log so large it pushes the root dir into AG 1.  We can't use -N for the mkfs
> +# because this check only occurs after the root directory has been allocated,
> +# which mkfs -N doesn't do.
> +test_format "log pushes rootdir into AG 1" -d agcount=3200,size=6366g -lagnum=0 -N

Why are you passing "-N" to the test if it can't be used to test
this?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 4/9] xfs: test xfs_copy doesn't do cached read before libxfs_mount

[Dave Chinner]

On Tue, Jun 28, 2022 at 01:21:40PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This is a regression test for an xfs_copy fix that ensures that it
> doesn't perform a cached read of an XFS filesystem prior to initializing
> libxfs, since the xfs_mount (and hence the buffer cache) isn't set up
> yet.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/844     |   33 +++++++++++++++++++++++++++++++++
>  tests/xfs/844.out |    3 +++
>  2 files changed, 36 insertions(+)
>  create mode 100755 tests/xfs/844
>  create mode 100644 tests/xfs/844.out
> 
> 
> diff --git a/tests/xfs/844 b/tests/xfs/844
> new file mode 100755
> index 00000000..688abe33
> --- /dev/null
> +++ b/tests/xfs/844
> @@ -0,0 +1,33 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test 844
> +#
> +# Regression test for xfsprogs commit:
> +#
> +# XXXXXXXX ("xfs_copy: don't use cached buffer reads until after libxfs_mount")
> +#

This needs more of an explanation of why empty files are being
copied here, because it's not obvious why we'd run xfs_copy on
them...

> +. ./common/preamble
> +_begin_fstest auto copy

Wouldn't this also be quick?

Otherwise looks fine.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 5/9] check: document mkfs.xfs reliance on fstests exports

[Dave Chinner]

On Tue, Jun 28, 2022 at 01:21:45PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> There are a number of fstests that employ special (and now unsupported)
> XFS filesystem configurations to perform testing in a controlled
> environment.  The presence of the QA_CHECK_FS and MSGVERB variables are
> used by mkfs.xfs to detect that it's running inside fstests, which
> enables the unsupported configurations.  Nobody else should be using
> filesystems with tiny logs, non-redundant superblocks, or smaller than
> the (new) minimum supported size.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

looks fine, and answers a question I had looking at some of the
mkfs modifications you've proposed.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 6/9] xfs/109: handle larger minimum filesystem size

[Dave Chinner]

On Tue, Jun 28, 2022 at 01:21:51PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> mkfs will soon refuse to format a filesystem smaller than 300MB, so
> increase the size of the filesystem to keep this test scenario
> realistic.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/109 |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> 
> diff --git a/tests/xfs/109 b/tests/xfs/109
> index 6cb6917a..e3e491f1 100755
> --- a/tests/xfs/109
> +++ b/tests/xfs/109
> @@ -78,7 +78,7 @@ if [ -n "$FASTSTART" -a -f $SCRATCH_MNT/f0 ]; then
>  fi
>  _scratch_unmount
>  
> -_scratch_mkfs_xfs -dsize=160m,agcount=4 $faststart | _filter_mkfs 2>$tmp.mkfs
> +_scratch_mkfs_xfs -dsize=320m,agcount=4 $faststart | _filter_mkfs 2>$tmp.mkfs
>  cat $tmp.mkfs >>$seqres.full
>  _scratch_mount

Looks fine.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 8/9] xfs/166: fix golden output failures when multipage folios enabled

[Dave Chinner]

On Tue, Jun 28, 2022 at 01:22:02PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Beginning with 5.18, some filesystems support creating large folios for
> the page cache.  A system with 64k pages can create 256k folios, which
> means that with the old file size of 1M, the last half of the file is
> completely converted from unwritten to written by page_mkwrite.  The
> test encodes a translated version of the xfs_bmap output in the golden
> output, which means that the test now fails on 64k pages.  Fixing the
> 64k page case by increasing the file size to 2MB broke fsdax because
> fsdax uses 2MB PMDs, hence 12MB.
> 
> Increase the size to prevent this from happening.  This may require
> further revision if folios get larger or fsdax starts supporting PMDs
> that are larger than 2MB.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/166 |   19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> 
> diff --git a/tests/xfs/166 b/tests/xfs/166
> index 42379961..d45dc5e8 100755
> --- a/tests/xfs/166
> +++ b/tests/xfs/166
> @@ -16,12 +16,12 @@ _begin_fstest rw metadata auto quick
>  # the others are unwritten.
>  _filter_blocks()
>  {
> -	$AWK_PROG '
> +	$AWK_PROG -v file_size=$FILE_SIZE '
>  /^ +[0-9]/ {
>  	if (!written_size) {
>  		written_size = $6
> -		unwritten1 = ((1048576/512) / 2) - written_size
> -		unwritten2 = ((1048576/512) / 2) - 2 * written_size
> +		unwritten1 = ((file_size/512) / 2) - written_size
> +		unwritten2 = ((file_size/512) / 2) - 2 * written_size
>  	}
>  
>  	# is the extent unwritten?
> @@ -58,7 +58,18 @@ _scratch_mount
>  
>  TEST_FILE=$SCRATCH_MNT/test_file
>  TEST_PROG=$here/src/unwritten_mmap
> -FILE_SIZE=1048576
> +
> +# Beginning with 5.18, some filesystems support creating large folios for the
> +# page cache.  A system with 64k pages can create 256k folios, which means
> +# that with the old file size of 1M, the last half of the file is completely
> +# converted from unwritten to written by page_mkwrite.  The test will fail on
> +# the golden output when this happens, so increase the size from the original
> +# 1MB file size to at least (6 * 256k == 1.5MB) prevent this from happening.
> +#
> +# However, increasing the file size to around 2MB causes regressions when fsdax
> +# is enabled because fsdax will try to use PMD entries for the mappings.  Hence
> +# we need to set the file size to (6 * 2MB == 12MB) to cover all cases.
> +FILE_SIZE=$((12 * 1048576))

LGTM.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 2/9] xfs/070: filter new superblock verifier messages

[Zorro Lang]

On Wed, Jun 29, 2022 at 02:15:47PM +1000, Dave Chinner wrote:
> On Tue, Jun 28, 2022 at 01:21:28PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > In Linux 5.19, the superblock verifier logging changed to elaborate on
> > what was wrong.  Fix the xfs_repair filtering function to accomodate
> > this development.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  common/repair |    1 +
> >  1 file changed, 1 insertion(+)
> > 
> > 
> > diff --git a/common/repair b/common/repair
> > index 463ef9db..398e9904 100644
> > --- a/common/repair
> > +++ b/common/repair
> > @@ -29,6 +29,7 @@ _filter_repair()
> >  # for sb
> >  /- agno = / && next;	# remove each AG line (variable number)
> >  s/(pointer to) (\d+)/\1 INO/;
> > +s/Superblock has bad magic number.*/bad magic number/;
> >  # Changed inode output in 5.5.0
> >  s/sb root inode value /sb root inode /;
> >  s/realtime bitmap inode value /realtime bitmap inode /;
> 
> Didn't I already fix that in commit 4c76d0ba ("xfs/070: filter the
> bad sb magic number error")?

Yes, you've added a line as below:
  s/^Superblock has (bad magic number) 0x.*/\1/;
which is equal to:
  s/Superblock has bad magic number.*/bad magic number/;
So we need to fix it again.

> 
> Cheers,
> 
> Dave.
> 
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 2/9] xfs/070: filter new superblock verifier messages

[Darrick J. Wong]

On Wed, Jun 29, 2022 at 05:12:48PM +0800, Zorro Lang wrote:
> On Wed, Jun 29, 2022 at 02:15:47PM +1000, Dave Chinner wrote:
> > On Tue, Jun 28, 2022 at 01:21:28PM -0700, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > In Linux 5.19, the superblock verifier logging changed to elaborate on
> > > what was wrong.  Fix the xfs_repair filtering function to accomodate
> > > this development.
> > > 
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  common/repair |    1 +
> > >  1 file changed, 1 insertion(+)
> > > 
> > > 
> > > diff --git a/common/repair b/common/repair
> > > index 463ef9db..398e9904 100644
> > > --- a/common/repair
> > > +++ b/common/repair
> > > @@ -29,6 +29,7 @@ _filter_repair()
> > >  # for sb
> > >  /- agno = / && next;	# remove each AG line (variable number)
> > >  s/(pointer to) (\d+)/\1 INO/;
> > > +s/Superblock has bad magic number.*/bad magic number/;
> > >  # Changed inode output in 5.5.0
> > >  s/sb root inode value /sb root inode /;
> > >  s/realtime bitmap inode value /realtime bitmap inode /;
> > 
> > Didn't I already fix that in commit 4c76d0ba ("xfs/070: filter the
> > bad sb magic number error")?

Ah whoops I guess we can drop this one then.

> Yes, you've added a line as below:
>   s/^Superblock has (bad magic number) 0x.*/\1/;
> which is equal to:
>   s/Superblock has bad magic number.*/bad magic number/;
> So we need to fix it again.

We .... do?

--D

> 
> > 
> > Cheers,
> > 
> > Dave.
> > 
> > -- 
> > Dave Chinner
> > david@fromorbit.com
> > 
> 

Re: [PATCH 3/9] xfs: test mkfs.xfs sizing of internal logs that

[Darrick J. Wong]

On Wed, Jun 29, 2022 at 02:18:07PM +1000, Dave Chinner wrote:
> On Tue, Jun 28, 2022 at 01:21:34PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > This is a regression test that exercises the mkfs.xfs code that creates
> > log sizes that are very close to the AG size when stripe units are in
> > play and/or when the log is forced to be in AG 0.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/843     |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/843.out |    2 ++
> >  2 files changed, 53 insertions(+)
> >  create mode 100755 tests/xfs/843
> >  create mode 100644 tests/xfs/843.out
> > 
> > 
> > diff --git a/tests/xfs/843 b/tests/xfs/843
> > new file mode 100755
> > index 00000000..5bb4bfb4
> > --- /dev/null
> > +++ b/tests/xfs/843
> > @@ -0,0 +1,51 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test 843
> > +#
> > +# Now that we've increased the default log size calculation, test mkfs with
> > +# various stripe units and filesystem sizes to see if we can provoke mkfs into
> > +# breaking.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto mkfs
> > +
> > +# real QA test starts here
> > +
> > +# Modify as appropriate.
> > +_supported_fs xfs
> > +_require_test
> > +echo Silence is golden
> > +
> > +testfile=$TEST_DIR/a
> > +rm -f $testfile
> > +
> > +test_format() {
> > +	local tag="$1"
> > +	shift
> > +
> > +	echo "$tag" >> $seqres.full
> > +	$MKFS_XFS_PROG $@ -d file,name=$testfile &>> $seqres.full
> > +	local res=$?
> > +	test $res -eq 0 || echo "$tag FAIL $res" | tee -a $seqres.full
> > +}
> > +
> > +# First we try various small filesystems and stripe sizes.
> > +for M in `seq 298 302` `seq 490 520`; do
> > +	for S in `seq 32 4 64`; do
> > +		test_format "M=$M S=$S" -dsu=${S}k,sw=1,size=${M}m -N
> > +	done
> > +done
> > +
> > +# Log so large it pushes the root dir into AG 1.  We can't use -N for the mkfs
> > +# because this check only occurs after the root directory has been allocated,
> > +# which mkfs -N doesn't do.
> > +test_format "log pushes rootdir into AG 1" -d agcount=3200,size=6366g -lagnum=0 -N
> 
> Why are you passing "-N" to the test if it can't be used to test
> this?

I guess I went a little overboard after you asked for more -N and less
test runtime last time.

--D

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH 4/9] xfs: test xfs_copy doesn't do cached read before libxfs_mount

[Darrick J. Wong]

On Wed, Jun 29, 2022 at 02:20:45PM +1000, Dave Chinner wrote:
> On Tue, Jun 28, 2022 at 01:21:40PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > This is a regression test for an xfs_copy fix that ensures that it
> > doesn't perform a cached read of an XFS filesystem prior to initializing
> > libxfs, since the xfs_mount (and hence the buffer cache) isn't set up
> > yet.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/844     |   33 +++++++++++++++++++++++++++++++++
> >  tests/xfs/844.out |    3 +++
> >  2 files changed, 36 insertions(+)
> >  create mode 100755 tests/xfs/844
> >  create mode 100644 tests/xfs/844.out
> > 
> > 
> > diff --git a/tests/xfs/844 b/tests/xfs/844
> > new file mode 100755
> > index 00000000..688abe33
> > --- /dev/null
> > +++ b/tests/xfs/844
> > @@ -0,0 +1,33 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test 844
> > +#
> > +# Regression test for xfsprogs commit:
> > +#
> > +# XXXXXXXX ("xfs_copy: don't use cached buffer reads until after libxfs_mount")
> > +#
> 
> This needs more of an explanation of why empty files are being
> copied here, because it's not obvious why we'd run xfs_copy on
> them...

Oops, I forgot to explain what this thing is actually testing.

> > +. ./common/preamble
> > +_begin_fstest auto copy
> 
> Wouldn't this also be quick?

Yeah.  Will fix both and resubmit.

--D

> Otherwise looks fine.
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

[PATCH v2.1 4/9] xfs: test xfs_copy doesn't do cached read before libxfs_mount

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test for an xfs_copy fix that ensures that it
doesn't perform a cached read of an XFS filesystem prior to initializing
libxfs, since the xfs_mount (and hence the buffer cache) isn't set up
yet.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/844     |   38 ++++++++++++++++++++++++++++++++++++++
 tests/xfs/844.out |    3 +++
 2 files changed, 41 insertions(+)
 create mode 100755 tests/xfs/844
 create mode 100644 tests/xfs/844.out

diff --git a/tests/xfs/844 b/tests/xfs/844
new file mode 100755
index 00000000..32349c85
--- /dev/null
+++ b/tests/xfs/844
@@ -0,0 +1,38 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test 844
+#
+# Regression test for xfsprogs commit:
+#
+# XXXXXXXX ("xfs_copy: don't use cached buffer reads until after libxfs_mount")
+#
+# It was discovered that passing xfs_copy a source device containing an ext4
+# filesystem would cause xfs_copy to crash.  Further investigation revealed
+# that any readable path that didn't have a plausible XFS superblock in block
+# zero would produce the same crash, so this regression test exploits that.
+#
+. ./common/preamble
+_begin_fstest auto copy quick
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_require_xfs_copy
+_require_test
+
+rm -f $TEST_DIR/$seq.*
+$XFS_IO_PROG -f -c 'truncate 100m' $TEST_DIR/$seq.a
+$XFS_IO_PROG -f -c 'truncate 100m' $TEST_DIR/$seq.b
+
+filter_copy() {
+	sed -e 's/Superblock has bad magic number.*/bad magic number/'
+}
+
+$XFS_COPY_PROG $TEST_DIR/$seq.a $TEST_DIR/$seq.b 2>&1 | filter_copy
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/844.out b/tests/xfs/844.out
new file mode 100644
index 00000000..dbefde1c
--- /dev/null
+++ b/tests/xfs/844.out
@@ -0,0 +1,3 @@
+QA output created by 844
+bad magic number
+xfs_copy: couldn't read superblock, error=22

Re: [PATCH 2/9] xfs/070: filter new superblock verifier messages

[Zorro Lang]

On Wed, Jun 29, 2022 at 03:41:10PM -0700, Darrick J. Wong wrote:
> On Wed, Jun 29, 2022 at 05:12:48PM +0800, Zorro Lang wrote:
> > On Wed, Jun 29, 2022 at 02:15:47PM +1000, Dave Chinner wrote:
> > > On Tue, Jun 28, 2022 at 01:21:28PM -0700, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > In Linux 5.19, the superblock verifier logging changed to elaborate on
> > > > what was wrong.  Fix the xfs_repair filtering function to accomodate
> > > > this development.
> > > > 
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > ---
> > > >  common/repair |    1 +
> > > >  1 file changed, 1 insertion(+)
> > > > 
> > > > 
> > > > diff --git a/common/repair b/common/repair
> > > > index 463ef9db..398e9904 100644
> > > > --- a/common/repair
> > > > +++ b/common/repair
> > > > @@ -29,6 +29,7 @@ _filter_repair()
> > > >  # for sb
> > > >  /- agno = / && next;	# remove each AG line (variable number)
> > > >  s/(pointer to) (\d+)/\1 INO/;
> > > > +s/Superblock has bad magic number.*/bad magic number/;
> > > >  # Changed inode output in 5.5.0
> > > >  s/sb root inode value /sb root inode /;
> > > >  s/realtime bitmap inode value /realtime bitmap inode /;
> > > 
> > > Didn't I already fix that in commit 4c76d0ba ("xfs/070: filter the
> > > bad sb magic number error")?
> 
> Ah whoops I guess we can drop this one then.
> 
> > Yes, you've added a line as below:
> >   s/^Superblock has (bad magic number) 0x.*/\1/;
> > which is equal to:
> >   s/Superblock has bad magic number.*/bad magic number/;
> > So we need to fix it again.
> 
> We .... do?

Sorry, "don't" missed :-P

> 
> --D
> 
> > 
> > > 
> > > Cheers,
> > > 
> > > Dave.
> > > 
> > > -- 
> > > Dave Chinner
> > > david@fromorbit.com
> > > 
> > 
> 

Re: [PATCH 3/9] xfs: test mkfs.xfs sizing of internal logs that

[Zorro Lang]

On Wed, Jun 29, 2022 at 03:43:36PM -0700, Darrick J. Wong wrote:
> On Wed, Jun 29, 2022 at 02:18:07PM +1000, Dave Chinner wrote:
> > On Tue, Jun 28, 2022 at 01:21:34PM -0700, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > This is a regression test that exercises the mkfs.xfs code that creates
> > > log sizes that are very close to the AG size when stripe units are in
> > > play and/or when the log is forced to be in AG 0.
> > > 
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  tests/xfs/843     |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
> > >  tests/xfs/843.out |    2 ++
> > >  2 files changed, 53 insertions(+)
> > >  create mode 100755 tests/xfs/843
> > >  create mode 100644 tests/xfs/843.out
> > > 
> > > 
> > > diff --git a/tests/xfs/843 b/tests/xfs/843
> > > new file mode 100755
> > > index 00000000..5bb4bfb4
> > > --- /dev/null
> > > +++ b/tests/xfs/843
> > > @@ -0,0 +1,51 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > > +#
> > > +# FS QA Test 843
> > > +#
> > > +# Now that we've increased the default log size calculation, test mkfs with
> > > +# various stripe units and filesystem sizes to see if we can provoke mkfs into
> > > +# breaking.
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest auto mkfs
> > > +
> > > +# real QA test starts here
> > > +
> > > +# Modify as appropriate.
> > > +_supported_fs xfs
> > > +_require_test
> > > +echo Silence is golden
> > > +
> > > +testfile=$TEST_DIR/a
> > > +rm -f $testfile
> > > +
> > > +test_format() {
> > > +	local tag="$1"
> > > +	shift
> > > +
> > > +	echo "$tag" >> $seqres.full
> > > +	$MKFS_XFS_PROG $@ -d file,name=$testfile &>> $seqres.full
> > > +	local res=$?
> > > +	test $res -eq 0 || echo "$tag FAIL $res" | tee -a $seqres.full
> > > +}
> > > +
> > > +# First we try various small filesystems and stripe sizes.
> > > +for M in `seq 298 302` `seq 490 520`; do
> > > +	for S in `seq 32 4 64`; do
> > > +		test_format "M=$M S=$S" -dsu=${S}k,sw=1,size=${M}m -N
> > > +	done
> > > +done
> > > +
> > > +# Log so large it pushes the root dir into AG 1.  We can't use -N for the mkfs
> > > +# because this check only occurs after the root directory has been allocated,
> > > +# which mkfs -N doesn't do.
> > > +test_format "log pushes rootdir into AG 1" -d agcount=3200,size=6366g -lagnum=0 -N
> > 
> > Why are you passing "-N" to the test if it can't be used to test
> > this?
> 
> I guess I went a little overboard after you asked for more -N and less
> test runtime last time.

Is there anything different coverage if mkfs.xfs with or without "-N" (except
really writing on disk) ? If no difference, I'm fine with that, or I think
without "-N" might be good, especially this case only makes fs with small size,
it's fast enough.

Thanks,
Zorro

> 
> --D
> 
> > Cheers,
> > 
> > Dave.
> > -- 
> > Dave Chinner
> > david@fromorbit.com
> 

Re: [PATCH 3/9] xfs: test mkfs.xfs sizing of internal logs that

[Darrick J. Wong]

On Thu, Jun 30, 2022 at 02:37:14PM +0800, Zorro Lang wrote:
> On Wed, Jun 29, 2022 at 03:43:36PM -0700, Darrick J. Wong wrote:
> > On Wed, Jun 29, 2022 at 02:18:07PM +1000, Dave Chinner wrote:
> > > On Tue, Jun 28, 2022 at 01:21:34PM -0700, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > This is a regression test that exercises the mkfs.xfs code that creates
> > > > log sizes that are very close to the AG size when stripe units are in
> > > > play and/or when the log is forced to be in AG 0.
> > > > 
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > ---
> > > >  tests/xfs/843     |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  tests/xfs/843.out |    2 ++
> > > >  2 files changed, 53 insertions(+)
> > > >  create mode 100755 tests/xfs/843
> > > >  create mode 100644 tests/xfs/843.out
> > > > 
> > > > 
> > > > diff --git a/tests/xfs/843 b/tests/xfs/843
> > > > new file mode 100755
> > > > index 00000000..5bb4bfb4
> > > > --- /dev/null
> > > > +++ b/tests/xfs/843
> > > > @@ -0,0 +1,51 @@
> > > > +#! /bin/bash
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > > > +#
> > > > +# FS QA Test 843
> > > > +#
> > > > +# Now that we've increased the default log size calculation, test mkfs with
> > > > +# various stripe units and filesystem sizes to see if we can provoke mkfs into
> > > > +# breaking.
> > > > +#
> > > > +. ./common/preamble
> > > > +_begin_fstest auto mkfs
> > > > +
> > > > +# real QA test starts here
> > > > +
> > > > +# Modify as appropriate.
> > > > +_supported_fs xfs
> > > > +_require_test
> > > > +echo Silence is golden
> > > > +
> > > > +testfile=$TEST_DIR/a
> > > > +rm -f $testfile
> > > > +
> > > > +test_format() {
> > > > +	local tag="$1"
> > > > +	shift
> > > > +
> > > > +	echo "$tag" >> $seqres.full
> > > > +	$MKFS_XFS_PROG $@ -d file,name=$testfile &>> $seqres.full
> > > > +	local res=$?
> > > > +	test $res -eq 0 || echo "$tag FAIL $res" | tee -a $seqres.full
> > > > +}
> > > > +
> > > > +# First we try various small filesystems and stripe sizes.
> > > > +for M in `seq 298 302` `seq 490 520`; do
> > > > +	for S in `seq 32 4 64`; do
> > > > +		test_format "M=$M S=$S" -dsu=${S}k,sw=1,size=${M}m -N
> > > > +	done
> > > > +done
> > > > +
> > > > +# Log so large it pushes the root dir into AG 1.  We can't use -N for the mkfs
> > > > +# because this check only occurs after the root directory has been allocated,
> > > > +# which mkfs -N doesn't do.
> > > > +test_format "log pushes rootdir into AG 1" -d agcount=3200,size=6366g -lagnum=0 -N
> > > 
> > > Why are you passing "-N" to the test if it can't be used to test
> > > this?
> > 
> > I guess I went a little overboard after you asked for more -N and less
> > test runtime last time.
> 
> Is there anything different coverage if mkfs.xfs with or without "-N" (except
> really writing on disk) ? If no difference, I'm fine with that, or I think
> without "-N" might be good, especially this case only makes fs with small size,
> it's fast enough.

Yes -- with that one exception, the mkfs geometry validation examined by
this test are all performed before mkfs writes anything to disk.  The
root inode location check is performed /after/ allocating it, which
means that we actually have to let it write the new filesystem to disk.

That's why *most* of the tests can use -N to reduce runtime.

Come to think of it, this really ought to have a _require_fs_space call
to make sure that fallocating the log space succeeds.

--D

> Thanks,
> Zorro
> 
> > 
> > --D
> > 
> > > Cheers,
> > > 
> > > Dave.
> > > -- 
> > > Dave Chinner
> > > david@fromorbit.com
> > 
> 

Re: [PATCH v2.1 4/9] xfs: test xfs_copy doesn't do cached read before libxfs_mount

[Dave Chinner]

On Wed, Jun 29, 2022 at 05:48:14PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This is a regression test for an xfs_copy fix that ensures that it
> doesn't perform a cached read of an XFS filesystem prior to initializing
> libxfs, since the xfs_mount (and hence the buffer cache) isn't set up
> yet.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCHSET 0/9] fstests: random fixes

[Zorro Lang]

On Tue, Jun 28, 2022 at 01:21:17PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> Here's the usual batch of odd fixes for fstests.
> 
> If you're going to start using this mess, you probably ought to just
> pull from my git trees, which are linked below.
> 
> This is an extraordinary way to destroy everything.  Enjoy!
> Comments and questions are, as always, welcome.
> 
> --D

Hi Darrick,

JFYI, to push the proceeding of your patch merging, I've merged/pushed this
patchset, except patch 7/9 and 9/9 due to they haven't gotten any review.
I can review them, but patch 7/9 changes the case which Dave might care about,
so I'd like to wait more response.

Feel free to send these 2 patches with more your new patches, if you'd like to
do that :)

Thanks,
Zorro

> 
> kernel git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes
> 
> xfsprogs git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes
> 
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
> ---
>  check                  |    3 +++
>  common/repair          |    1 +
>  src/seek_sanity_test.c |   12 ++++++++++-
>  tests/xfs/018          |   52 +++++++++++++++++++++++++++++++++++++++++++-----
>  tests/xfs/018.out      |   16 ++++-----------
>  tests/xfs/109          |    2 +-
>  tests/xfs/166          |   19 ++++++++++++++----
>  tests/xfs/547          |   14 +++++++++----
>  tests/xfs/843          |   51 +++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/843.out      |    2 ++
>  tests/xfs/844          |   33 ++++++++++++++++++++++++++++++
>  tests/xfs/844.out      |    3 +++
>  12 files changed, 181 insertions(+), 27 deletions(-)
>  create mode 100755 tests/xfs/843
>  create mode 100644 tests/xfs/843.out
>  create mode 100755 tests/xfs/844
>  create mode 100644 tests/xfs/844.out
> 

Re: [PATCHSET 0/9] fstests: random fixes

[Darrick J. Wong]

On Tue, Jul 05, 2022 at 10:28:40PM +0800, Zorro Lang wrote:
> On Tue, Jun 28, 2022 at 01:21:17PM -0700, Darrick J. Wong wrote:
> > Hi all,
> > 
> > Here's the usual batch of odd fixes for fstests.
> > 
> > If you're going to start using this mess, you probably ought to just
> > pull from my git trees, which are linked below.
> > 
> > This is an extraordinary way to destroy everything.  Enjoy!
> > Comments and questions are, as always, welcome.
> > 
> > --D
> 
> Hi Darrick,
> 
> JFYI, to push the proceeding of your patch merging, I've merged/pushed this
> patchset, except patch 7/9 and 9/9 due to they haven't gotten any review.
> I can review them, but patch 7/9 changes the case which Dave might care about,
> so I'd like to wait more response.
> 
> Feel free to send these 2 patches with more your new patches, if you'd like to
> do that :)

Done.  Thanks for taking last week's bundle, btw.

--D

> Thanks,
> Zorro
> 
> > 
> > kernel git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes
> > 
> > xfsprogs git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes
> > 
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
> > ---
> >  check                  |    3 +++
> >  common/repair          |    1 +
> >  src/seek_sanity_test.c |   12 ++++++++++-
> >  tests/xfs/018          |   52 +++++++++++++++++++++++++++++++++++++++++++-----
> >  tests/xfs/018.out      |   16 ++++-----------
> >  tests/xfs/109          |    2 +-
> >  tests/xfs/166          |   19 ++++++++++++++----
> >  tests/xfs/547          |   14 +++++++++----
> >  tests/xfs/843          |   51 +++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/843.out      |    2 ++
> >  tests/xfs/844          |   33 ++++++++++++++++++++++++++++++
> >  tests/xfs/844.out      |    3 +++
> >  12 files changed, 181 insertions(+), 27 deletions(-)
> >  create mode 100755 tests/xfs/843
> >  create mode 100644 tests/xfs/843.out
> >  create mode 100755 tests/xfs/844
> >  create mode 100644 tests/xfs/844.out
> > 
> 

Re: [PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[liuyd.fnst]

Hi, guys.

Recently I hit a regression during test xfstest. Reverting this commit 
could fix that error.

Reproduce steps (only nfs4.2 has this error):
```
# cat local.config

export TEST_DEV=127.0.0.1:/home/nfs/share0

export TEST_DIR=/mnt/test

export SCRATCH_DEV=127.0.0.1:/home/nfs/share1

export SCRATCH_MNT=/mnt/scratch

export FSX_AVOID="-E"

export NFS_MOUNT_OPTIONS="-o rw,relatime,vers=4.2"


# ./check -nfs generic/285

FSTYP         -- nfs

PLATFORM      -- Linux/aarch64 hpe-apollo80-01-n00 
5.14.0-131.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Jul 18 16:13:44 EDT 2022

MKFS_OPTIONS  -- 127.0.0.1:/home/nfs/share1

MOUNT_OPTIONS -- -o rw,relatime,vers=4.2 -o 
context=system_u:object_r:root_t:s0 127.0.0.1:/home/nfs/share1 /mnt/scratch



generic/285 2s ... [failed, exit status 1]- output mismatch (see 
/root/xfstests/results//generic/285.out.bad)

     --- tests/generic/285.out	2022-07-27 21:07:43.160268552 -0400

     +++ /root/xfstests/results//generic/285.out.bad	2022-07-27 
21:31:27.887090532 -0400

     @@ -1 +1,3 @@

      QA output created by 285

     +seek sanity check failed!

     +(see /root/xfstests/results//generic/285.full for details)

     ...

     (Run 'diff -u /root/xfstests/tests/generic/285.out 
/root/xfstests/results//generic/285.out.bad'  to see the entire diff)

Ran: generic/285

Failures: generic/285

Failed 1 of 1 tests


```

Reverting this commit then test pass.
```
# git revert e861a30255c9780425ee5193325d30882fbe7410
# make -j && make install -j
---snip---
# ./check -nfs generic/285

FSTYP         -- nfs

PLATFORM      -- Linux/aarch64 hpe-apollo80-01-n00 
5.14.0-131.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Jul 18 16:13:44 EDT 2022

MKFS_OPTIONS  -- 127.0.0.1:/home/nfs/share1

MOUNT_OPTIONS -- -o rw,relatime,vers=4.2 -o 
context=system_u:object_r:root_t:s0 127.0.0.1:/home/nfs/share1 /mnt/scratch



generic/285 1s ...  1s

Ran: generic/285

Passed all 1 tests

```

On 6/29/22 04:21, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> The seek sanity test tries to figure out a file space allocation unit by
> calling stat and then using an iterative SEEK_DATA method to try to
> detect a smaller blocksize based on SEEK_DATA's consultation of the
> filesystem's internal block mapping.  This was put in (AFAICT) because
> XFS' stat implementation returns max(filesystem blocksize, PAGESIZE) for
> most regular files.
> 
> Unfortunately, for a realtime file with an extent size larger than a
> single filesystem block this doesn't work at all because block mappings
> still work at filesystem block granularity, but allocation units do not.
> To fix this, detect the specific case where st_blksize != PAGE_SIZE and
> trust the fstat results.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>   src/seek_sanity_test.c |   12 +++++++++++-
>   1 file changed, 11 insertions(+), 1 deletion(-)
> 
> 
> diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
> index 76587b7f..1030d0c5 100644
> --- a/src/seek_sanity_test.c
> +++ b/src/seek_sanity_test.c
> @@ -45,6 +45,7 @@ static int get_io_sizes(int fd)
>   	off_t pos = 0, offset = 1;
>   	struct stat buf;
>   	int shift, ret;
> +	int pagesz = sysconf(_SC_PAGE_SIZE);
>   
>   	ret = fstat(fd, &buf);
>   	if (ret) {
> @@ -53,8 +54,16 @@ static int get_io_sizes(int fd)
>   		return ret;
>   	}
>   
> -	/* st_blksize is typically also the allocation size */
> +	/*
> +	 * st_blksize is typically also the allocation size.  However, XFS
> +	 * rounds this up to the page size, so if the stat blocksize is exactly
> +	 * one page, use this iterative algorithm to see if SEEK_DATA will hint
> +	 * at a more precise answer based on the filesystem's (pre)allocation
> +	 * decisions.
> +	 */
>   	alloc_size = buf.st_blksize;
> +	if (alloc_size != pagesz)
> +		goto done;
>   
>   	/* try to discover the actual alloc size */
>   	while (pos == 0 && offset < alloc_size) {
> @@ -80,6 +89,7 @@ static int get_io_sizes(int fd)
>   	if (!shift)
>   		offset += pos ? 0 : 1;
>   	alloc_size = offset;
> +done:
>   	fprintf(stdout, "Allocation size: %ld\n", alloc_size);
>   	return 0;
>   
> 

Re: [PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[Darrick J. Wong]

On Thu, Jul 28, 2022 at 01:37:08AM +0000, liuyd.fnst@fujitsu.com wrote:
> Hi, guys.
> 
> Recently I hit a regression during test xfstest. Reverting this commit 
> could fix that error.
> 
> Reproduce steps (only nfs4.2 has this error):
> ```
> # cat local.config
> 
> export TEST_DEV=127.0.0.1:/home/nfs/share0
> 
> export TEST_DIR=/mnt/test
> 
> export SCRATCH_DEV=127.0.0.1:/home/nfs/share1

ahahaaaa, NFS.  I forgot that it sets st_blksize == 1048576, which will
trip this up.

Let me work on a fix to seek_sanity_test.c to make this work for xfs
realtime without breaking NFS.  I /think/ the solution is to replace the
"if (...) goto done;" bit with something that queries FS_IOC_GETXATTR
and XFS_IOC_FSGEOMETRY to figure out if the file being tested is a
realtime file on an XFS filesystem, and set alloc_size to the rt extent
size.

/* Compute the file allocation unit size for an XFS file. */
static int detect_xfs_alloc_unit(int fd)
{
	struct fsxattr fsx;
	struct xfs_fsop_geom fsgeom;
	int ret;

	ret = ioctl(fd, XFS_IOC_FSGEOMETRY, &fsgeom);
	if (ret)
		return -1;

	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fsx);
	if (ret)
		return -1;

	alloc_size = fsgeom.blocksize;
	if (fsx.fsx_xflags & FS_XFLAG_REALTIME)
		alloc_size *= fsgeom.rtextsize;

	return 0;
}

should suffice to fix these testcase on xfs realtime without messing up
nfs.

--D

> export SCRATCH_MNT=/mnt/scratch
> 
> export FSX_AVOID="-E"
> 
> export NFS_MOUNT_OPTIONS="-o rw,relatime,vers=4.2"
> 
> 
> # ./check -nfs generic/285
> 
> FSTYP         -- nfs
> 
> PLATFORM      -- Linux/aarch64 hpe-apollo80-01-n00 
> 5.14.0-131.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Jul 18 16:13:44 EDT 2022
> 
> MKFS_OPTIONS  -- 127.0.0.1:/home/nfs/share1
> 
> MOUNT_OPTIONS -- -o rw,relatime,vers=4.2 -o 
> context=system_u:object_r:root_t:s0 127.0.0.1:/home/nfs/share1 /mnt/scratch
> 
> 
> 
> generic/285 2s ... [failed, exit status 1]- output mismatch (see 
> /root/xfstests/results//generic/285.out.bad)
> 
>      --- tests/generic/285.out	2022-07-27 21:07:43.160268552 -0400
> 
>      +++ /root/xfstests/results//generic/285.out.bad	2022-07-27 
> 21:31:27.887090532 -0400
> 
>      @@ -1 +1,3 @@
> 
>       QA output created by 285
> 
>      +seek sanity check failed!
> 
>      +(see /root/xfstests/results//generic/285.full for details)
> 
>      ...
> 
>      (Run 'diff -u /root/xfstests/tests/generic/285.out 
> /root/xfstests/results//generic/285.out.bad'  to see the entire diff)
> 
> Ran: generic/285
> 
> Failures: generic/285
> 
> Failed 1 of 1 tests
> 
> 
> ```
> 
> Reverting this commit then test pass.
> ```
> # git revert e861a30255c9780425ee5193325d30882fbe7410
> # make -j && make install -j
> ---snip---
> # ./check -nfs generic/285
> 
> FSTYP         -- nfs
> 
> PLATFORM      -- Linux/aarch64 hpe-apollo80-01-n00 
> 5.14.0-131.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Jul 18 16:13:44 EDT 2022
> 
> MKFS_OPTIONS  -- 127.0.0.1:/home/nfs/share1
> 
> MOUNT_OPTIONS -- -o rw,relatime,vers=4.2 -o 
> context=system_u:object_r:root_t:s0 127.0.0.1:/home/nfs/share1 /mnt/scratch
> 
> 
> 
> generic/285 1s ...  1s
> 
> Ran: generic/285
> 
> Passed all 1 tests
> 
> ```
> 
> On 6/29/22 04:21, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > The seek sanity test tries to figure out a file space allocation unit by
> > calling stat and then using an iterative SEEK_DATA method to try to
> > detect a smaller blocksize based on SEEK_DATA's consultation of the
> > filesystem's internal block mapping.  This was put in (AFAICT) because
> > XFS' stat implementation returns max(filesystem blocksize, PAGESIZE) for
> > most regular files.
> > 
> > Unfortunately, for a realtime file with an extent size larger than a
> > single filesystem block this doesn't work at all because block mappings
> > still work at filesystem block granularity, but allocation units do not.
> > To fix this, detect the specific case where st_blksize != PAGE_SIZE and
> > trust the fstat results.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >   src/seek_sanity_test.c |   12 +++++++++++-
> >   1 file changed, 11 insertions(+), 1 deletion(-)
> > 
> > 
> > diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
> > index 76587b7f..1030d0c5 100644
> > --- a/src/seek_sanity_test.c
> > +++ b/src/seek_sanity_test.c
> > @@ -45,6 +45,7 @@ static int get_io_sizes(int fd)
> >   	off_t pos = 0, offset = 1;
> >   	struct stat buf;
> >   	int shift, ret;
> > +	int pagesz = sysconf(_SC_PAGE_SIZE);
> >   
> >   	ret = fstat(fd, &buf);
> >   	if (ret) {
> > @@ -53,8 +54,16 @@ static int get_io_sizes(int fd)
> >   		return ret;
> >   	}
> >   
> > -	/* st_blksize is typically also the allocation size */
> > +	/*
> > +	 * st_blksize is typically also the allocation size.  However, XFS
> > +	 * rounds this up to the page size, so if the stat blocksize is exactly
> > +	 * one page, use this iterative algorithm to see if SEEK_DATA will hint
> > +	 * at a more precise answer based on the filesystem's (pre)allocation
> > +	 * decisions.
> > +	 */
> >   	alloc_size = buf.st_blksize;
> > +	if (alloc_size != pagesz)
> > +		goto done;
> >   
> >   	/* try to discover the actual alloc size */
> >   	while (pos == 0 && offset < alloc_size) {
> > @@ -80,6 +89,7 @@ static int get_io_sizes(int fd)
> >   	if (!shift)
> >   		offset += pos ? 0 : 1;
> >   	alloc_size = offset;
> > +done:
> >   	fprintf(stdout, "Allocation size: %ld\n", alloc_size);
> >   	return 0;
> >   
> > 

Re: [PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[Darrick J. Wong]

Does this patch fix NFS for you?

--D
---
From: Darrick J. Wong <djwong@kernel.org>

seek_sanity_test: use XFS ioctls to determine file allocation unit size

liuyd.fnst@fujitsu.com reported that my recent change to the seek sanity
test broke NFS.  I foolishly thought that st_blksize was sufficient to
find the file allocation unit size so that applications could figure out
the SEEK_HOLE granularity.  Replace that with an explicit callout to XFS
ioctls so that xfs realtime will work again.

Fixes: e861a302 ("seek_sanity_test: fix allocation unit detection on XFS realtime")
Reported-by: liuyd.fnst@fujitsu.com
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 src/Makefile           |    4 ++++
 src/seek_sanity_test.c |   40 +++++++++++++++++++++++++++++++---------
 2 files changed, 35 insertions(+), 9 deletions(-)

diff --git a/src/Makefile b/src/Makefile
index 38628a22..b89a7a5e 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -82,6 +82,10 @@ ifeq ($(HAVE_LIBCAP), true)
 LLDLIBS += -lcap
 endif
 
+ifeq ($(HAVE_FSXATTR_XFLAG_HASATTR), yes)
+LCFLAGS += -DHAVE_FSXATTR_XFLAG_HASATTR
+endif
+
 ifeq ($(HAVE_SEEK_DATA), yes)
  ifeq ($(HAVE_FSXATTR_XFLAG_HASATTR), yes)
   ifeq ($(HAVE_NFTW), yes)
diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
index 1030d0c5..b53f4862 100644
--- a/src/seek_sanity_test.c
+++ b/src/seek_sanity_test.c
@@ -40,6 +40,32 @@ static void get_file_system(int fd)
 	}
 }
 
+#ifdef HAVE_FSXATTR_XFLAG_HASATTR
+/* Compute the file allocation unit size for an XFS file. */
+static int detect_xfs_alloc_unit(int fd)
+{
+	struct fsxattr fsx;
+	struct xfs_fsop_geom fsgeom;
+	int ret;
+
+	ret = ioctl(fd, XFS_IOC_FSGEOMETRY, &fsgeom);
+	if (ret)
+		return -1;
+
+	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fsx);
+	if (ret)
+		return -1;
+
+	alloc_size = fsgeom.blocksize;
+	if (fsx.fsx_xflags & FS_XFLAG_REALTIME)
+		alloc_size *= fsgeom.rtextsize;
+
+	return 0;
+}
+#else
+# define detect_xfs_alloc_unit(fd) (-1)
+#endif
+
 static int get_io_sizes(int fd)
 {
 	off_t pos = 0, offset = 1;
@@ -47,6 +73,10 @@ static int get_io_sizes(int fd)
 	int shift, ret;
 	int pagesz = sysconf(_SC_PAGE_SIZE);
 
+	ret = detect_xfs_alloc_unit(fd);
+	if (!ret)
+		goto done;
+
 	ret = fstat(fd, &buf);
 	if (ret) {
 		fprintf(stderr, "  ERROR %d: Failed to find io blocksize\n",
@@ -54,16 +84,8 @@ static int get_io_sizes(int fd)
 		return ret;
 	}
 
-	/*
-	 * st_blksize is typically also the allocation size.  However, XFS
-	 * rounds this up to the page size, so if the stat blocksize is exactly
-	 * one page, use this iterative algorithm to see if SEEK_DATA will hint
-	 * at a more precise answer based on the filesystem's (pre)allocation
-	 * decisions.
-	 */
+	/* st_blksize is typically also the allocation size */
 	alloc_size = buf.st_blksize;
-	if (alloc_size != pagesz)
-		goto done;
 
 	/* try to discover the actual alloc size */
 	while (pos == 0 && offset < alloc_size) {

Re: [PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[liuyd.fnst]

Hi, Darrick.


On 7/28/22 10:37, Darrick J. Wong wrote:
> Does this patch fix NFS for you?

It works for me. Thanks.
BTW. It has conflict during "git am". Looks like your branch are ahead 
of master.

> 
> --D
> ---
> From: Darrick J. Wong <djwong@kernel.org>
> 
> seek_sanity_test: use XFS ioctls to determine file allocation unit size
> 
> liuyd.fnst@fujitsu.com reported that my recent change to the seek sanity
> test broke NFS.  I foolishly thought that st_blksize was sufficient to
> find the file allocation unit size so that applications could figure out
> the SEEK_HOLE granularity.  Replace that with an explicit callout to XFS
> ioctls so that xfs realtime will work again.
> 
> Fixes: e861a302 ("seek_sanity_test: fix allocation unit detection on XFS realtime")
> Reported-by: liuyd.fnst@fujitsu.com
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>   src/Makefile           |    4 ++++
>   src/seek_sanity_test.c |   40 +++++++++++++++++++++++++++++++---------
>   2 files changed, 35 insertions(+), 9 deletions(-)
> 
> diff --git a/src/Makefile b/src/Makefile
> index 38628a22..b89a7a5e 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -82,6 +82,10 @@ ifeq ($(HAVE_LIBCAP), true)
>   LLDLIBS += -lcap
>   endif
>   
> +ifeq ($(HAVE_FSXATTR_XFLAG_HASATTR), yes)
> +LCFLAGS += -DHAVE_FSXATTR_XFLAG_HASATTR
> +endif
> +
>   ifeq ($(HAVE_SEEK_DATA), yes)
>    ifeq ($(HAVE_FSXATTR_XFLAG_HASATTR), yes)
>     ifeq ($(HAVE_NFTW), yes)
> diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
> index 1030d0c5..b53f4862 100644
> --- a/src/seek_sanity_test.c
> +++ b/src/seek_sanity_test.c
> @@ -40,6 +40,32 @@ static void get_file_system(int fd)
>   	}
>   }
>   
> +#ifdef HAVE_FSXATTR_XFLAG_HASATTR
> +/* Compute the file allocation unit size for an XFS file. */
> +static int detect_xfs_alloc_unit(int fd)
> +{
> +	struct fsxattr fsx;
> +	struct xfs_fsop_geom fsgeom;
> +	int ret;
> +
> +	ret = ioctl(fd, XFS_IOC_FSGEOMETRY, &fsgeom);
> +	if (ret)
> +		return -1;
> +
> +	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fsx);
> +	if (ret)
> +		return -1;
> +
> +	alloc_size = fsgeom.blocksize;
> +	if (fsx.fsx_xflags & FS_XFLAG_REALTIME)
> +		alloc_size *= fsgeom.rtextsize;
> +
> +	return 0;
> +}
> +#else
> +# define detect_xfs_alloc_unit(fd) (-1)
> +#endif
> +
>   static int get_io_sizes(int fd)
>   {
>   	off_t pos = 0, offset = 1;
> @@ -47,6 +73,10 @@ static int get_io_sizes(int fd)
>   	int shift, ret;
>   	int pagesz = sysconf(_SC_PAGE_SIZE);
>   
> +	ret = detect_xfs_alloc_unit(fd);
> +	if (!ret)
> +		goto done;
> +
>   	ret = fstat(fd, &buf);
>   	if (ret) {
>   		fprintf(stderr, "  ERROR %d: Failed to find io blocksize\n",
> @@ -54,16 +84,8 @@ static int get_io_sizes(int fd)
>   		return ret;
>   	}
>   
> -	/*
> -	 * st_blksize is typically also the allocation size.  However, XFS
> -	 * rounds this up to the page size, so if the stat blocksize is exactly
> -	 * one page, use this iterative algorithm to see if SEEK_DATA will hint
> -	 * at a more precise answer based on the filesystem's (pre)allocation
> -	 * decisions.
> -	 */
> +	/* st_blksize is typically also the allocation size */
>   	alloc_size = buf.st_blksize;
> -	if (alloc_size != pagesz)
> -		goto done;
>   
>   	/* try to discover the actual alloc size */
>   	while (pos == 0 && offset < alloc_size) {

Re: [PATCH 1/9] seek_sanity_test: fix allocation unit detection on XFS realtime

[Darrick J. Wong]

On Thu, Jul 28, 2022 at 03:38:38AM +0000, liuyd.fnst@fujitsu.com wrote:
> Hi, Darrick.
> 
> 
> On 7/28/22 10:37, Darrick J. Wong wrote:
> > Does this patch fix NFS for you?
> 
> It works for me. Thanks.
> BTW. It has conflict during "git am". Looks like your branch are ahead 
> of master.

Yes, I base my development branch off of for-next nowadays.  Thanks for
the report, I'll add a tested-by if that's ok and send it out as a bug
fix.

--D

> > 
> > --D
> > ---
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > seek_sanity_test: use XFS ioctls to determine file allocation unit size
> > 
> > liuyd.fnst@fujitsu.com reported that my recent change to the seek sanity
> > test broke NFS.  I foolishly thought that st_blksize was sufficient to
> > find the file allocation unit size so that applications could figure out
> > the SEEK_HOLE granularity.  Replace that with an explicit callout to XFS
> > ioctls so that xfs realtime will work again.
> > 
> > Fixes: e861a302 ("seek_sanity_test: fix allocation unit detection on XFS realtime")
> > Reported-by: liuyd.fnst@fujitsu.com
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >   src/Makefile           |    4 ++++
> >   src/seek_sanity_test.c |   40 +++++++++++++++++++++++++++++++---------
> >   2 files changed, 35 insertions(+), 9 deletions(-)
> > 
> > diff --git a/src/Makefile b/src/Makefile
> > index 38628a22..b89a7a5e 100644
> > --- a/src/Makefile
> > +++ b/src/Makefile
> > @@ -82,6 +82,10 @@ ifeq ($(HAVE_LIBCAP), true)
> >   LLDLIBS += -lcap
> >   endif
> >   
> > +ifeq ($(HAVE_FSXATTR_XFLAG_HASATTR), yes)
> > +LCFLAGS += -DHAVE_FSXATTR_XFLAG_HASATTR
> > +endif
> > +
> >   ifeq ($(HAVE_SEEK_DATA), yes)
> >    ifeq ($(HAVE_FSXATTR_XFLAG_HASATTR), yes)
> >     ifeq ($(HAVE_NFTW), yes)
> > diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
> > index 1030d0c5..b53f4862 100644
> > --- a/src/seek_sanity_test.c
> > +++ b/src/seek_sanity_test.c
> > @@ -40,6 +40,32 @@ static void get_file_system(int fd)
> >   	}
> >   }
> >   
> > +#ifdef HAVE_FSXATTR_XFLAG_HASATTR
> > +/* Compute the file allocation unit size for an XFS file. */
> > +static int detect_xfs_alloc_unit(int fd)
> > +{
> > +	struct fsxattr fsx;
> > +	struct xfs_fsop_geom fsgeom;
> > +	int ret;
> > +
> > +	ret = ioctl(fd, XFS_IOC_FSGEOMETRY, &fsgeom);
> > +	if (ret)
> > +		return -1;
> > +
> > +	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fsx);
> > +	if (ret)
> > +		return -1;
> > +
> > +	alloc_size = fsgeom.blocksize;
> > +	if (fsx.fsx_xflags & FS_XFLAG_REALTIME)
> > +		alloc_size *= fsgeom.rtextsize;
> > +
> > +	return 0;
> > +}
> > +#else
> > +# define detect_xfs_alloc_unit(fd) (-1)
> > +#endif
> > +
> >   static int get_io_sizes(int fd)
> >   {
> >   	off_t pos = 0, offset = 1;
> > @@ -47,6 +73,10 @@ static int get_io_sizes(int fd)
> >   	int shift, ret;
> >   	int pagesz = sysconf(_SC_PAGE_SIZE);
> >   
> > +	ret = detect_xfs_alloc_unit(fd);
> > +	if (!ret)
> > +		goto done;
> > +
> >   	ret = fstat(fd, &buf);
> >   	if (ret) {
> >   		fprintf(stderr, "  ERROR %d: Failed to find io blocksize\n",
> > @@ -54,16 +84,8 @@ static int get_io_sizes(int fd)
> >   		return ret;
> >   	}
> >   
> > -	/*
> > -	 * st_blksize is typically also the allocation size.  However, XFS
> > -	 * rounds this up to the page size, so if the stat blocksize is exactly
> > -	 * one page, use this iterative algorithm to see if SEEK_DATA will hint
> > -	 * at a more precise answer based on the filesystem's (pre)allocation
> > -	 * decisions.
> > -	 */
> > +	/* st_blksize is typically also the allocation size */
> >   	alloc_size = buf.st_blksize;
> > -	if (alloc_size != pagesz)
> > -		goto done;
> >   
> >   	/* try to discover the actual alloc size */
> >   	while (pos == 0 && offset < alloc_size) {

Re: [PATCHSET 0/9] fstests: random fixes

[Eryu Guan]

On Tue, Apr 13, 2021 at 06:04:30PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> This branch contains fixes to various tests to fix miscellaneous test
> bugs and unnecessary regressions when XFS is configured with somewhat
> unusual configurations (e.g. always-cow mode, external logs, and/or
> realtime devices).

Thanks for all the fixes! I've applied all patches except the 5th one.

Thanks,
Eryu

[PATCHSET 0/9] fstests: random fixes

[Darrick J. Wong]

Hi all,

This branch contains fixes to various tests to fix miscellaneous test
bugs and unnecessary regressions when XFS is configured with somewhat
unusual configurations (e.g. always-cow mode, external logs, and/or
realtime devices).

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
 common/dmthin     |    9 ++++++++-
 common/dump       |    1 +
 common/rc         |   13 ++++++++++++-
 common/repair     |    2 +-
 tests/generic/094 |    5 +++++
 tests/generic/223 |    3 +++
 tests/generic/225 |    5 +++++
 tests/generic/347 |    2 +-
 tests/generic/500 |    2 +-
 tests/generic/563 |    7 ++++++-
 tests/generic/620 |    2 +-
 tests/xfs/030     |    2 +-
 tests/xfs/083     |    2 +-
 tests/xfs/305     |    2 +-
 tests/xfs/506     |    1 +
 tests/xfs/521     |    2 +-
 tests/xfs/530     |    2 +-
 17 files changed, 50 insertions(+), 12 deletions(-)