[PATCH 0/7] fstests: fix quota failures on xfs

[PATCH 0/7] fstests: fix quota failures on xfs

[Darrick J. Wong]

Hi all,

Several months ago I tried testing xfs with quotas enabled and was
surprised at the number of failures.  Since then I've been on a bug hunt
to triage the mess, and the following series fixes some of the low
hanging fruit.  Some of these are minor behavioral variations when quota
are enabled, and other fixes upgrade old v4 tests to work on v5
filesystems.

To reproduce the results, run xfstests with
'MOUNT_OPTIONS="-o usrquota,grpquota,prjquota"' on a v5 filesystem.
Strike the ',grpquota' on a v4 filesystem.

The first patch fixes differences in the golden output when quotas are
enabled on xfs -- mmap writes (intentionally) crashing with SIGBUS when
quota updates fail vs. msync failing with EIO.

The second patch fixes all the tests that require a specific quota mount
option configuration to filter out any quota options present in
MOUNT_OPTIONS.

The third patch updates xfs/001 to work with XFS v5 filesystems since
there was nothing particularly v4 specific about it.

The fourth patch enables tests that require a v4 format to filter out v5
mount options (such as grpquota) so that they can run.

Patch #5 upgrades xfs/288 to support XFS v5 since there was nothing
particular to v4 about that test.

Patch #6 fixes a problem in generic/338 where switching the disk backend
to dm-error on a xfs with quotas enabled causes the fs to go down with a
dirty log which is not recovered prior to the end of the test.  This was
originally a regression test for a NULL pointer dereference after the fs
shuts down, so we can skip the post-test fsck.

Patch #7 filters 'quota info will be regenerated on next quota mount'
messages from the xfs_repair golden output.

--D

[PATCH 1/7] generic/{279, 28[1-3]}: hide SIGBUS reporting from golden output

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

These four tests check that mmap'd cow writes fail when the filesystem
goes down.  For regular filesystems the msync reports EIO, but if quotas
are enabled on xfs the write itself terminates xfs_io with a SIGBUS.  We
don't care how the write fails, so don't let the SIGBUS report escape to
the golden output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/279 |    6 ++++++
 tests/generic/281 |    6 ++++++
 tests/generic/282 |    9 ++++++++-
 tests/generic/283 |   14 ++++++++++----
 4 files changed, 30 insertions(+), 5 deletions(-)


diff --git a/tests/generic/279 b/tests/generic/279
index 1355c641..af699144 100755
--- a/tests/generic/279
+++ b/tests/generic/279
@@ -66,8 +66,14 @@ md5sum $testdir/file2 | _filter_scratch
 echo "CoW and unmount"
 sync
 _dmerror_load_error_table
+# Insulate ourselves against bash reporting the SIGBUS when we try to modify
+# the metadata.
+cat > $tmp.run << ENDL
+ulimit -c 0
 $XFS_IO_PROG -f -c "mmap -rw 0 $filesize" -c "mwrite -S 0x63 0 $filesize" \
 	-c "msync -s 0 $filesize" $testdir/file2 >> $seqres.full 2>&1
+ENDL
+bash $tmp.run 2> /dev/null
 _dmerror_load_working_table
 _dmerror_unmount
 _dmerror_mount
diff --git a/tests/generic/281 b/tests/generic/281
index 75c4b39a..75deab7c 100755
--- a/tests/generic/281
+++ b/tests/generic/281
@@ -66,8 +66,14 @@ md5sum $testdir/file2 | _filter_scratch
 echo "CoW and unmount"
 sync
 _dmerror_load_error_table
+# Insulate ourselves against bash reporting the SIGBUS when we try to modify
+# the metadata.
+cat > $tmp.run << ENDL
+ulimit -c 0
 $XFS_IO_PROG -f -c "mmap -rw 0 $filesize" -c "mwrite -S 0x63 0 $filesize" \
 	-c "msync -s 0 $filesize" $testdir/file2 >> $seqres.full 2>&1
+ENDL
+bash $tmp.run 2> /dev/null
 
 echo "Clean up the mess"
 _dmerror_unmount
diff --git a/tests/generic/282 b/tests/generic/282
index aea7ce58..44ffe278 100755
--- a/tests/generic/282
+++ b/tests/generic/282
@@ -66,7 +66,14 @@ md5sum $testdir/file2 | _filter_scratch
 echo "CoW and unmount"
 sync
 _dmerror_load_error_table
-$XFS_IO_PROG -f -c "mmap -rw 0 $filesize" -c "mwrite -S 0x63 0 $filesize" $testdir/file2 >> $seqres.full 2>&1
+# Insulate ourselves against bash reporting the SIGBUS when we try to modify
+# the metadata.
+cat > $tmp.run << ENDL
+ulimit -c 0
+$XFS_IO_PROG -f -c "mmap -rw 0 $filesize" \
+	-c "mwrite -S 0x63 0 $filesize" $testdir/file2 >> $seqres.full 2>&1
+ENDL
+bash $tmp.run 2> /dev/null
 _dmerror_load_working_table
 rm -rf $testdir/file2 >> $seqres.full 2>&1
 _dmerror_unmount
diff --git a/tests/generic/283 b/tests/generic/283
index 79caddb5..84a1666a 100755
--- a/tests/generic/283
+++ b/tests/generic/283
@@ -67,10 +67,16 @@ md5sum $testdir/file2 | _filter_scratch
 echo "CoW and unmount"
 sync
 _dmerror_load_error_table
-urk=$($XFS_IO_PROG -f -c "mmap -rw 0 $filesize" -c "mwrite -S 0x63 0 $filesize" \
-	-c "msync -s 0 $filesize" $testdir/file2 2>&1)
-echo $urk >> $seqres.full
-echo "$urk" | grep -q "error" || _fail "mwrite did not fail"
+# Insulate ourselves against bash reporting the SIGBUS when we try to modify
+# the metadata.
+cat > $tmp.run << ENDL
+ulimit -c 0
+$XFS_IO_PROG -f -c "mmap -rw 0 $filesize" -c "mwrite -S 0x63 0 $filesize" \
+	-c "msync -s 0 $filesize" $testdir/file2 2>&1
+ENDL
+bash $tmp.run > $tmp.output 2>&1
+cat $tmp.output >> $seqres.full
+grep -q error $tmp.output || _fail "mwrite did not fail"
 _dmerror_load_working_table
 
 echo "Rewrite"

[PATCH 2/7] misc: force the exact quota options coded into the test

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Several tests require setting exactly the quota mount options that are
encoded in the test.  Unfortunately, quota mount options set by the user
in MOUNT_OPTIONS can turn on other quota types, which causes the golden
outputs to be off for various reasons (quota report output, behavior
changes, etc.)

So use _qmount_option to delete all the quota mount options in these
tests so that the test itself can run with exactly the config for which
it was written.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/230 |    3 +++
 tests/xfs/075     |    4 ++++
 tests/xfs/116     |    3 +++
 tests/xfs/132     |    5 +++++
 tests/xfs/220     |    3 +++
 5 files changed, 18 insertions(+)


diff --git a/tests/generic/230 b/tests/generic/230
index bc8c1ad0..a7ab67f7 100755
--- a/tests/generic/230
+++ b/tests/generic/230
@@ -99,6 +99,9 @@ rm -f $seqres.full
 
 grace=2
 
+# Test only the quota type specified in each _scratch_mount clause below.
+_qmount_option 'defaults'
+
 _scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount "-o usrquota,grpquota"
 quotacheck -u -g $SCRATCH_MNT 2>/dev/null
diff --git a/tests/xfs/075 b/tests/xfs/075
index caf612bb..dee01462 100755
--- a/tests/xfs/075
+++ b/tests/xfs/075
@@ -25,12 +25,16 @@ _cleanup()
 # get standard environment, filters and checks
 . ./common/rc
 . ./common/filter
+. ./common/quota
 
 # real QA test starts here
 _supported_fs xfs
 _supported_os Linux
 _require_scratch
 
+# norecovery mounts do not succeed with quotas eanbled, so shut them off
+_qmount_option "defaults"
+
 echo "Silence is golden"
 _scratch_mkfs_sized $((512 * 1024 * 1024)) >$seqres.full
 
diff --git a/tests/xfs/116 b/tests/xfs/116
index 6887d7e5..4422eb50 100755
--- a/tests/xfs/116
+++ b/tests/xfs/116
@@ -36,6 +36,9 @@ _supported_os Linux
 _require_scratch
 _require_xfs_quota
 
+# Only mount with the quota options we specify below
+_qmount_option "defaults"
+
 _scratch_mkfs >/dev/null 2>&1
 _scratch_mount "-o uquota"
 _scratch_unmount
diff --git a/tests/xfs/132 b/tests/xfs/132
index c5fe8b5a..a421f727 100755
--- a/tests/xfs/132
+++ b/tests/xfs/132
@@ -24,6 +24,7 @@ _cleanup()
 # get standard environment, filters and checks
 . ./common/rc
 . ./common/filter
+. ./common/quota
 
 # remove previous $seqres.full before test
 rm -f $seqres.full
@@ -43,6 +44,10 @@ _disable_dmesg_check
 _require_xfs_mkfs_crc
 _scratch_mkfs -m crc=0 > $seqres.full 2>&1
 
+# The files that EIO in the golden output changes if we have quotas enabled
+# so leave it off.
+_qmount_option "defaults"
+
 # corrupt an inode in the root inode chunk
 root_ino=$(_scratch_xfs_get_metadata_field 'rootino' 'sb 0')
 corrupt_ino=$((root_ino + 15))
diff --git a/tests/xfs/220 b/tests/xfs/220
index 227d5452..61319c9d 100755
--- a/tests/xfs/220
+++ b/tests/xfs/220
@@ -36,6 +36,9 @@ _supported_os Linux
 _require_scratch
 _require_quota
 
+# Only mount with the specific quota options mentioned below
+_qmount_option "defaults"
+
 echo "Silence is golden."
 
 # create scratch filesystem

[PATCH 3/7] xfs/001: update to handle v5 filesystems

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

xfs/001 fuzzes various extent fields using xfs_db.  There's nothing in
it that's specific to v4 filesystems, so upgrade the test to be able to
handle v5 filesystems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/xfs    |   15 +++++++++++++++
 tests/xfs/001 |   18 ++++++++++++------
 2 files changed, 27 insertions(+), 6 deletions(-)


diff --git a/common/xfs b/common/xfs
index 9a785b88..5b632df1 100644
--- a/common/xfs
+++ b/common/xfs
@@ -765,3 +765,18 @@ _scratch_get_sfdir_prefix() {
 	_scratch_xfs_db -c "inode ${dir_ino}" -c 'p' >> $seqres.full
 	return 1
 }
+
+_scratch_get_bmx_prefix() {
+	local ino="$1"
+
+	for prefix in "u3.bmx" "u.bmx"; do
+		if [ -n "$(_scratch_xfs_get_metadata_field \
+				"${prefix}[0].startblock" \
+				"inode ${ino}")" ]; then
+			echo "${prefix}"
+			return 0
+		fi
+	done
+	_scratch_xfs_db -c "inode ${ino}" -c 'p' >> $seqres.full
+	return 1
+}
diff --git a/tests/xfs/001 b/tests/xfs/001
index ac86c3b1..6ad3aba8 100755
--- a/tests/xfs/001
+++ b/tests/xfs/001
@@ -49,9 +49,8 @@ _do_bit_test()
 _supported_fs xfs
 _supported_os Linux
 _require_scratch_nocheck
-_require_xfs_mkfs_crc
 
-_scratch_mkfs -m crc=0 >/dev/null 2>&1
+_scratch_mkfs >/dev/null 2>&1
 _scratch_mount
 
 # create the test file
@@ -68,11 +67,18 @@ BMBT_STARTOFF_BITLEN=54
 BMBT_STARTBLOCK_BITLEN=52
 BMBT_BLOCKCOUNT_BITLEN=21
 
+prefix=$(_scratch_get_bmx_prefix $FILE_INO)
+test -n "$prefix" || _fail "could not find bmx prefix from inode $FILE_INO"
+
+filter_output() {
+	sed -e "s/${prefix}/u.bmx/g"
+}
+
 # test setting the BMBT entries from 0 to past the valid number.
-_do_bit_test "u.bmx[0].extentflag" $BMBT_EXNTFLAG_BITLEN
-_do_bit_test "u.bmx[0].startoff" $BMBT_STARTOFF_BITLEN
-_do_bit_test "u.bmx[0].startblock" $BMBT_STARTBLOCK_BITLEN
-_do_bit_test "u.bmx[0].blockcount" $BMBT_BLOCKCOUNT_BITLEN
+_do_bit_test "${prefix}[0].extentflag" $BMBT_EXNTFLAG_BITLEN | filter_output
+_do_bit_test "${prefix}[0].startoff" $BMBT_STARTOFF_BITLEN | filter_output
+_do_bit_test "${prefix}[0].startblock" $BMBT_STARTBLOCK_BITLEN | filter_output
+_do_bit_test "${prefix}[0].blockcount" $BMBT_BLOCKCOUNT_BITLEN | filter_output
 # test setting the 32 bit generation number
 _scratch_xfs_db -x -c "inode $FILE_INO" -c "write core.gen 0x5a"
 _scratch_xfs_db -x -c "inode $FILE_INO" -c "write core.gen 0xa5"

[PATCH 4/7] xfs: filter out mount options that don't work on v4 filesystems

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

A few tests require v4 filesystems and enforce this by disabling crc's
in the _scratch_mkfs call.  However, if the user specified MOUNT_OPTIONS
that only work with v5 filesystems, these tests fail.  Filter out
incompatible mount options that don't work on v4, such as simultaneous
group/project quota.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/xfs    |   23 +++++++++++++++++++++++
 tests/xfs/002 |    2 ++
 tests/xfs/194 |    2 ++
 tests/xfs/199 |    2 ++
 4 files changed, 29 insertions(+)


diff --git a/common/xfs b/common/xfs
index 5b632df1..33d819d7 100644
--- a/common/xfs
+++ b/common/xfs
@@ -780,3 +780,26 @@ _scratch_get_bmx_prefix() {
 	_scratch_xfs_db -c "inode ${ino}" -c 'p' >> $seqres.full
 	return 1
 }
+
+#
+# Ensures that we don't pass any mount options incompatible with XFS v4
+#
+_filter_xfsv4_mount_options()
+{
+	local gquota=0
+	local pquota=0
+
+	# Can't have group and project quotas in XFS v4
+	echo "$MOUNT_OPTIONS" | egrep -q "(gquota|grpquota|grpjquota=|gqnoenforce)" && gquota=1
+	echo "$MOUNT_OPTIONS" | egrep -q "(\bpquota|prjquota|pqnoenforce)" && pquota=1
+
+	if [ $gquota -gt 0 ] && [ $pquota -gt 0 ]; then
+		export MOUNT_OPTIONS=$(echo $MOUNT_OPTIONS \
+			| sed   -e 's/gquota/QUOTA/g'      \
+				-e 's/grpquota/QUOTA/g'    \
+				-e 's/grpjquota=[^, ]/QUOTA/g' \
+				-e 's/gqnoenforce/QUOTA/g' \
+				-e "s/QUOTA/defaults/g")
+	fi
+	echo "MOUNT_OPTIONS = $MOUNT_OPTIONS" >>$seqres.full
+}
diff --git a/tests/xfs/002 b/tests/xfs/002
index bd1c0a7a..f987575c 100755
--- a/tests/xfs/002
+++ b/tests/xfs/002
@@ -45,6 +45,8 @@ rm -f $seqres.full
 
 _scratch_mkfs_xfs -m crc=0 -d size=128m >> $seqres.full 2>&1 || _fail "mkfs failed"
 
+_filter_xfsv4_mount_options
+
 # Scribble past a couple V4 secondary superblocks to populate sb_crc
 # (We can't write to the structure member because it doesn't exist
 # on a v4 superblock, so we use the data type & "write fill")
diff --git a/tests/xfs/194 b/tests/xfs/194
index 6c1eddba..09a3840d 100755
--- a/tests/xfs/194
+++ b/tests/xfs/194
@@ -74,6 +74,8 @@ _require_scratch
 unset MKFS_OPTIONS
 unset XFS_MKFS_OPTIONS
 
+_filter_xfsv4_mount_options
+
 # we need 512 byte block size, so crc's are turned off
 _scratch_mkfs_xfs -m crc=0 -b size=$blksize >/dev/null 2>&1
 _scratch_mount
diff --git a/tests/xfs/199 b/tests/xfs/199
index dd909b09..54fe54fe 100755
--- a/tests/xfs/199
+++ b/tests/xfs/199
@@ -34,6 +34,8 @@ _supported_os Linux
 
 _require_scratch
 
+_filter_xfsv4_mount_options
+
 # clear any mkfs options so that we can directly specify the options we need to
 # be able to test the features bitmask behaviour correctly.
 MKFS_OPTIONS=

[PATCH 5/7] xfs/288: update for v5 filesystem support in xfs_db

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

The xfs_db 'write -d' command has supported v5 filesystems for a few
releases now, and there's nothing about this test that require v4, so
let the test run with v5 if the user so specifies.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/288 |    8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)


diff --git a/tests/xfs/288 b/tests/xfs/288
index e7a08a5d..bccdb6fc 100755
--- a/tests/xfs/288
+++ b/tests/xfs/288
@@ -36,14 +36,8 @@ _supported_os Linux
 _require_scratch
 _require_attrs
 
-# Due to xfs_db's write -d option is still not stable, and there's no
-# plan to support attr3. So only run this case on V4 XFS.
-# Please update this if xfs_db get enough improvement in one day.
-if [ -z "$XFS_MKFS_HAS_NO_META_SUPPORT" ]; then
-	mkfs_opts="-m crc=0"
-fi
 # get block size ($dbsize) from the mkfs output
-_scratch_mkfs_xfs $mkfs_opts 2>/dev/null | _filter_mkfs 2>$tmp.mkfs >/dev/null
+_scratch_mkfs_xfs 2>/dev/null | _filter_mkfs 2>$tmp.mkfs >/dev/null
 . $tmp.mkfs
 
 _scratch_mount

[PATCH 6/7] generic/338: don't check fs after crashing it

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

This test tries to make the filesystem go down by setting up dm-error
and committing metadata updates.  Since the test doesn't remount the fs
after it goes down to recover the log, this can result in a dirty log
being presented to the post-test filesystem check if the filesystem is
xfs and quotas are enabled.

Since this is a regression test for a NULL pointer dereference in the
kernel after the fs goes down, simply skip the post-test fsck.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/338 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/generic/338 b/tests/generic/338
index 6d701e83..81122e2d 100755
--- a/tests/generic/338
+++ b/tests/generic/338
@@ -36,7 +36,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs generic
 _supported_os Linux
-_require_scratch
+_require_scratch_nocheck # fs went down with a dirty log, don't check it
 _require_dm_target error
 # If SCRATCH_DEV is not a valid block device, FSTYP cannot be mkfs'ed either
 _require_block_device $SCRATCH_DEV

[PATCH 7/7] misc: filter out quota regeneration messages

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Filter out quota regeneration messages from xfs_repair when we check the
filesystem, because the xfs tests that encode xfs_repair output in the
golden output will fail when quotas are enabled and the xfs_repair quota
messages appear.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/repair |    1 +
 1 file changed, 1 insertion(+)


diff --git a/common/repair b/common/repair
index 6bbaffbe..c94939c9 100644
--- a/common/repair
+++ b/common/repair
@@ -81,6 +81,7 @@ s/(superblock) (\d+)/\1 AGNO/;
 s/(AG \#)(\d+)/\1AGNO/;
 s/(reset bad sb for ag) (\d+)/\1 AGNO/;
 s/(unknown block state, ag )(\d+)(, block )(\d+)/\1AGNO\3AGBNO/;
+/^Note - quota info will be regenerated on next quota mount.$/ && next;
 	print;'
 }
 

Re: [PATCH 2/7] misc: force the exact quota options coded into the test

[Eryu Guan]

On Tue, Jul 03, 2018 at 09:51:00PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Several tests require setting exactly the quota mount options that are
> encoded in the test.  Unfortunately, quota mount options set by the user
> in MOUNT_OPTIONS can turn on other quota types, which causes the golden
> outputs to be off for various reasons (quota report output, behavior
> changes, etc.)
> 
> So use _qmount_option to delete all the quota mount options in these
> tests so that the test itself can run with exactly the config for which
> it was written.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  tests/generic/230 |    3 +++
>  tests/xfs/075     |    4 ++++
>  tests/xfs/116     |    3 +++
>  tests/xfs/132     |    5 +++++
>  tests/xfs/220     |    3 +++
>  5 files changed, 18 insertions(+)
> 
> 
> diff --git a/tests/generic/230 b/tests/generic/230
> index bc8c1ad0..a7ab67f7 100755
> --- a/tests/generic/230
> +++ b/tests/generic/230
> @@ -99,6 +99,9 @@ rm -f $seqres.full
>  
>  grace=2
>  
> +# Test only the quota type specified in each _scratch_mount clause below.
> +_qmount_option 'defaults'
> +

IMHO, instead of converting all user defined quota related options to
'defaults' then mount with _scratch_mount, it's better to convert them
to what we want and mount with _qmount, i.e.

_qmount_option "usrquota,grpquota"
_qmount

This is the standard process for quota tests.

>  _scratch_mkfs >> $seqres.full 2>&1
>  _scratch_mount "-o usrquota,grpquota"
>  quotacheck -u -g $SCRATCH_MNT 2>/dev/null

Also, with _qmount we could remove all these quotacheck and quotaon.

Thanks,
Eryu

> diff --git a/tests/xfs/075 b/tests/xfs/075
> index caf612bb..dee01462 100755
> --- a/tests/xfs/075
> +++ b/tests/xfs/075
> @@ -25,12 +25,16 @@ _cleanup()
>  # get standard environment, filters and checks
>  . ./common/rc
>  . ./common/filter
> +. ./common/quota
>  
>  # real QA test starts here
>  _supported_fs xfs
>  _supported_os Linux
>  _require_scratch
>  
> +# norecovery mounts do not succeed with quotas eanbled, so shut them off
> +_qmount_option "defaults"
> +
>  echo "Silence is golden"
>  _scratch_mkfs_sized $((512 * 1024 * 1024)) >$seqres.full
>  
> diff --git a/tests/xfs/116 b/tests/xfs/116
> index 6887d7e5..4422eb50 100755
> --- a/tests/xfs/116
> +++ b/tests/xfs/116
> @@ -36,6 +36,9 @@ _supported_os Linux
>  _require_scratch
>  _require_xfs_quota
>  
> +# Only mount with the quota options we specify below
> +_qmount_option "defaults"
> +
>  _scratch_mkfs >/dev/null 2>&1
>  _scratch_mount "-o uquota"
>  _scratch_unmount
> diff --git a/tests/xfs/132 b/tests/xfs/132
> index c5fe8b5a..a421f727 100755
> --- a/tests/xfs/132
> +++ b/tests/xfs/132
> @@ -24,6 +24,7 @@ _cleanup()
>  # get standard environment, filters and checks
>  . ./common/rc
>  . ./common/filter
> +. ./common/quota
>  
>  # remove previous $seqres.full before test
>  rm -f $seqres.full
> @@ -43,6 +44,10 @@ _disable_dmesg_check
>  _require_xfs_mkfs_crc
>  _scratch_mkfs -m crc=0 > $seqres.full 2>&1
>  
> +# The files that EIO in the golden output changes if we have quotas enabled
> +# so leave it off.
> +_qmount_option "defaults"
> +
>  # corrupt an inode in the root inode chunk
>  root_ino=$(_scratch_xfs_get_metadata_field 'rootino' 'sb 0')
>  corrupt_ino=$((root_ino + 15))
> diff --git a/tests/xfs/220 b/tests/xfs/220
> index 227d5452..61319c9d 100755
> --- a/tests/xfs/220
> +++ b/tests/xfs/220
> @@ -36,6 +36,9 @@ _supported_os Linux
>  _require_scratch
>  _require_quota
>  
> +# Only mount with the specific quota options mentioned below
> +_qmount_option "defaults"
> +
>  echo "Silence is golden."
>  
>  # create scratch filesystem
> 

Re: [PATCH 4/7] xfs: filter out mount options that don't work on v4 filesystems

[Eryu Guan]

On Tue, Jul 03, 2018 at 09:51:12PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> A few tests require v4 filesystems and enforce this by disabling crc's
> in the _scratch_mkfs call.  However, if the user specified MOUNT_OPTIONS
> that only work with v5 filesystems, these tests fail.  Filter out
> incompatible mount options that don't work on v4, such as simultaneous
> group/project quota.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  common/xfs    |   23 +++++++++++++++++++++++
>  tests/xfs/002 |    2 ++
>  tests/xfs/194 |    2 ++
>  tests/xfs/199 |    2 ++
>  4 files changed, 29 insertions(+)
> 
> 
> diff --git a/common/xfs b/common/xfs
> index 5b632df1..33d819d7 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -780,3 +780,26 @@ _scratch_get_bmx_prefix() {
>  	_scratch_xfs_db -c "inode ${ino}" -c 'p' >> $seqres.full
>  	return 1
>  }
> +
> +#
> +# Ensures that we don't pass any mount options incompatible with XFS v4
> +#
> +_filter_xfsv4_mount_options()

"_filter" usually indicates a filter function that gets input from a
pipe, e.g. <some cmd> | _filter_scratch, but this helper doesn't follow
the convention. It'd better to rename it to ... I don't know, just
_xfsv4_mount_options?

> +{
> +	local gquota=0
> +	local pquota=0
> +
> +	# Can't have group and project quotas in XFS v4
> +	echo "$MOUNT_OPTIONS" | egrep -q "(gquota|grpquota|grpjquota=|gqnoenforce)" && gquota=1
> +	echo "$MOUNT_OPTIONS" | egrep -q "(\bpquota|prjquota|pqnoenforce)" && pquota=1
> +
> +	if [ $gquota -gt 0 ] && [ $pquota -gt 0 ]; then
> +		export MOUNT_OPTIONS=$(echo $MOUNT_OPTIONS \
> +			| sed   -e 's/gquota/QUOTA/g'      \
> +				-e 's/grpquota/QUOTA/g'    \
> +				-e 's/grpjquota=[^, ]/QUOTA/g' \
> +				-e 's/gqnoenforce/QUOTA/g' \
> +				-e "s/QUOTA/defaults/g")
> +	fi
> +	echo "MOUNT_OPTIONS = $MOUNT_OPTIONS" >>$seqres.full
> +}
> diff --git a/tests/xfs/002 b/tests/xfs/002
> index bd1c0a7a..f987575c 100755
> --- a/tests/xfs/002
> +++ b/tests/xfs/002
> @@ -45,6 +45,8 @@ rm -f $seqres.full
>  
>  _scratch_mkfs_xfs -m crc=0 -d size=128m >> $seqres.full 2>&1 || _fail "mkfs failed"
>  
> +_filter_xfsv4_mount_options

Hmm, this way future test writers have to remember to add a
_filter_xfsv4_mount_options call everytime after making a v4 xfs. It's
so easy to get lost and hard to maintain.

How about make _scratch_mkfs_xfs do the work automatically if it finds
the newly created fs is a v4 xfs?


And I went through all tests that have explicit "-m crc=0" mkfs option,
and I found that there're some other tests need fix too.

xfs/045, it doesn't fail, but the test expects a mount failure anyway,
just that it's caused by quota options now.

xfs/132, it fails to mount xfs and needs the fix. But it still fails
even after the fix, but that's a different issue.

xfs/300, the test itself passes, it just causes subsequent
_scratch_mount to fail in _check_xfs_filesystem(). I think we could just
umount $SCRATCH_DEV explicitly after test, so _check_xfs_filesystem()
won't try to mount it again.

Thanks,
Eryu

> +
>  # Scribble past a couple V4 secondary superblocks to populate sb_crc
>  # (We can't write to the structure member because it doesn't exist
>  # on a v4 superblock, so we use the data type & "write fill")
> diff --git a/tests/xfs/194 b/tests/xfs/194
> index 6c1eddba..09a3840d 100755
> --- a/tests/xfs/194
> +++ b/tests/xfs/194
> @@ -74,6 +74,8 @@ _require_scratch
>  unset MKFS_OPTIONS
>  unset XFS_MKFS_OPTIONS
>  
> +_filter_xfsv4_mount_options
> +
>  # we need 512 byte block size, so crc's are turned off
>  _scratch_mkfs_xfs -m crc=0 -b size=$blksize >/dev/null 2>&1
>  _scratch_mount
> diff --git a/tests/xfs/199 b/tests/xfs/199
> index dd909b09..54fe54fe 100755
> --- a/tests/xfs/199
> +++ b/tests/xfs/199
> @@ -34,6 +34,8 @@ _supported_os Linux
>  
>  _require_scratch
>  
> +_filter_xfsv4_mount_options
> +
>  # clear any mkfs options so that we can directly specify the options we need to
>  # be able to test the features bitmask behaviour correctly.
>  MKFS_OPTIONS=
> 

Re: [PATCH 0/7] fstests: fix quota failures on xfs

[Eryu Guan]

On Tue, Jul 03, 2018 at 09:50:47PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> Several months ago I tried testing xfs with quotas enabled and was
> surprised at the number of failures.  Since then I've been on a bug hunt
> to triage the mess, and the following series fixes some of the low
> hanging fruit.  Some of these are minor behavioral variations when quota
> are enabled, and other fixes upgrade old v4 tests to work on v5
> filesystems.

(Sorry for the late review..)

I take parts of the patchset (patch 1 3 5-7) for this week's update, as
the patches don't have hard dependencies. I dropped patch 2 and 4 for
now.

Thanks,
Eryu

> 
> To reproduce the results, run xfstests with
> 'MOUNT_OPTIONS="-o usrquota,grpquota,prjquota"' on a v5 filesystem.
> Strike the ',grpquota' on a v4 filesystem.
> 
> The first patch fixes differences in the golden output when quotas are
> enabled on xfs -- mmap writes (intentionally) crashing with SIGBUS when
> quota updates fail vs. msync failing with EIO.
> 
> The second patch fixes all the tests that require a specific quota mount
> option configuration to filter out any quota options present in
> MOUNT_OPTIONS.
> 
> The third patch updates xfs/001 to work with XFS v5 filesystems since
> there was nothing particularly v4 specific about it.
> 
> The fourth patch enables tests that require a v4 format to filter out v5
> mount options (such as grpquota) so that they can run.
> 
> Patch #5 upgrades xfs/288 to support XFS v5 since there was nothing
> particular to v4 about that test.
> 
> Patch #6 fixes a problem in generic/338 where switching the disk backend
> to dm-error on a xfs with quotas enabled causes the fs to go down with a
> dirty log which is not recovered prior to the end of the test.  This was
> originally a regression test for a NULL pointer dereference after the fs
> shuts down, so we can skip the post-test fsck.
> 
> Patch #7 filters 'quota info will be regenerated on next quota mount'
> messages from the xfs_repair golden output.
> 
> --D

Re: [PATCH 0/7] fstests: fix quota failures on xfs

[Darrick J. Wong]

On Sat, Jul 14, 2018 at 07:49:35PM +0800, Eryu Guan wrote:
> On Tue, Jul 03, 2018 at 09:50:47PM -0700, Darrick J. Wong wrote:
> > Hi all,
> > 
> > Several months ago I tried testing xfs with quotas enabled and was
> > surprised at the number of failures.  Since then I've been on a bug hunt
> > to triage the mess, and the following series fixes some of the low
> > hanging fruit.  Some of these are minor behavioral variations when quota
> > are enabled, and other fixes upgrade old v4 tests to work on v5
> > filesystems.
> 
> (Sorry for the late review..)
> 
> I take parts of the patchset (patch 1 3 5-7) for this week's update, as
> the patches don't have hard dependencies. I dropped patch 2 and 4 for
> now.

Ok, thank you!  I'll work on 2 and 4 for next time. :)

--D

> Thanks,
> Eryu
> 
> > 
> > To reproduce the results, run xfstests with
> > 'MOUNT_OPTIONS="-o usrquota,grpquota,prjquota"' on a v5 filesystem.
> > Strike the ',grpquota' on a v4 filesystem.
> > 
> > The first patch fixes differences in the golden output when quotas are
> > enabled on xfs -- mmap writes (intentionally) crashing with SIGBUS when
> > quota updates fail vs. msync failing with EIO.
> > 
> > The second patch fixes all the tests that require a specific quota mount
> > option configuration to filter out any quota options present in
> > MOUNT_OPTIONS.
> > 
> > The third patch updates xfs/001 to work with XFS v5 filesystems since
> > there was nothing particularly v4 specific about it.
> > 
> > The fourth patch enables tests that require a v4 format to filter out v5
> > mount options (such as grpquota) so that they can run.
> > 
> > Patch #5 upgrades xfs/288 to support XFS v5 since there was nothing
> > particular to v4 about that test.
> > 
> > Patch #6 fixes a problem in generic/338 where switching the disk backend
> > to dm-error on a xfs with quotas enabled causes the fs to go down with a
> > dirty log which is not recovered prior to the end of the test.  This was
> > originally a regression test for a NULL pointer dereference after the fs
> > shuts down, so we can skip the post-test fsck.
> > 
> > Patch #7 filters 'quota info will be regenerated on next quota mount'
> > messages from the xfs_repair golden output.
> > 
> > --D
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/7] xfs: filter out mount options that don't work on v4 filesystems

[Darrick J. Wong]

On Sat, Jul 14, 2018 at 07:41:28PM +0800, Eryu Guan wrote:
> On Tue, Jul 03, 2018 at 09:51:12PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > 
> > A few tests require v4 filesystems and enforce this by disabling crc's
> > in the _scratch_mkfs call.  However, if the user specified MOUNT_OPTIONS
> > that only work with v5 filesystems, these tests fail.  Filter out
> > incompatible mount options that don't work on v4, such as simultaneous
> > group/project quota.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  common/xfs    |   23 +++++++++++++++++++++++
> >  tests/xfs/002 |    2 ++
> >  tests/xfs/194 |    2 ++
> >  tests/xfs/199 |    2 ++
> >  4 files changed, 29 insertions(+)
> > 
> > 
> > diff --git a/common/xfs b/common/xfs
> > index 5b632df1..33d819d7 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -780,3 +780,26 @@ _scratch_get_bmx_prefix() {
> >  	_scratch_xfs_db -c "inode ${ino}" -c 'p' >> $seqres.full
> >  	return 1
> >  }
> > +
> > +#
> > +# Ensures that we don't pass any mount options incompatible with XFS v4
> > +#
> > +_filter_xfsv4_mount_options()
> 
> "_filter" usually indicates a filter function that gets input from a
> pipe, e.g. <some cmd> | _filter_scratch, but this helper doesn't follow
> the convention. It'd better to rename it to ... I don't know, just
> _xfsv4_mount_options?

_force_xfsv4_mount_options?

> > +{
> > +	local gquota=0
> > +	local pquota=0
> > +
> > +	# Can't have group and project quotas in XFS v4
> > +	echo "$MOUNT_OPTIONS" | egrep -q "(gquota|grpquota|grpjquota=|gqnoenforce)" && gquota=1
> > +	echo "$MOUNT_OPTIONS" | egrep -q "(\bpquota|prjquota|pqnoenforce)" && pquota=1
> > +
> > +	if [ $gquota -gt 0 ] && [ $pquota -gt 0 ]; then
> > +		export MOUNT_OPTIONS=$(echo $MOUNT_OPTIONS \
> > +			| sed   -e 's/gquota/QUOTA/g'      \
> > +				-e 's/grpquota/QUOTA/g'    \
> > +				-e 's/grpjquota=[^, ]/QUOTA/g' \
> > +				-e 's/gqnoenforce/QUOTA/g' \
> > +				-e "s/QUOTA/defaults/g")
> > +	fi
> > +	echo "MOUNT_OPTIONS = $MOUNT_OPTIONS" >>$seqres.full
> > +}
> > diff --git a/tests/xfs/002 b/tests/xfs/002
> > index bd1c0a7a..f987575c 100755
> > --- a/tests/xfs/002
> > +++ b/tests/xfs/002
> > @@ -45,6 +45,8 @@ rm -f $seqres.full
> >  
> >  _scratch_mkfs_xfs -m crc=0 -d size=128m >> $seqres.full 2>&1 || _fail "mkfs failed"
> >  
> > +_filter_xfsv4_mount_options
> 
> Hmm, this way future test writers have to remember to add a
> _filter_xfsv4_mount_options call everytime after making a v4 xfs. It's
> so easy to get lost and hard to maintain.
> 
> How about make _scratch_mkfs_xfs do the work automatically if it finds
> the newly created fs is a v4 xfs?

Sure, sounds good to me.

> And I went through all tests that have explicit "-m crc=0" mkfs option,
> and I found that there're some other tests need fix too.
> 
> xfs/045, it doesn't fail, but the test expects a mount failure anyway,
> just that it's caused by quota options now.
> 
> xfs/132, it fails to mount xfs and needs the fix. But it still fails
> even after the fix, but that's a different issue.

I think this is fixed by a subsequent patch which ... eh, this thread
was from too long ago, I'll just resend the whole thing.

> xfs/300, the test itself passes, it just causes subsequent
> _scratch_mount to fail in _check_xfs_filesystem(). I think we could just
> umount $SCRATCH_DEV explicitly after test, so _check_xfs_filesystem()
> won't try to mount it again.

Hm, I'll take a look at that one too.

--D

> Thanks,
> Eryu
> 
> > +
> >  # Scribble past a couple V4 secondary superblocks to populate sb_crc
> >  # (We can't write to the structure member because it doesn't exist
> >  # on a v4 superblock, so we use the data type & "write fill")
> > diff --git a/tests/xfs/194 b/tests/xfs/194
> > index 6c1eddba..09a3840d 100755
> > --- a/tests/xfs/194
> > +++ b/tests/xfs/194
> > @@ -74,6 +74,8 @@ _require_scratch
> >  unset MKFS_OPTIONS
> >  unset XFS_MKFS_OPTIONS
> >  
> > +_filter_xfsv4_mount_options
> > +
> >  # we need 512 byte block size, so crc's are turned off
> >  _scratch_mkfs_xfs -m crc=0 -b size=$blksize >/dev/null 2>&1
> >  _scratch_mount
> > diff --git a/tests/xfs/199 b/tests/xfs/199
> > index dd909b09..54fe54fe 100755
> > --- a/tests/xfs/199
> > +++ b/tests/xfs/199
> > @@ -34,6 +34,8 @@ _supported_os Linux
> >  
> >  _require_scratch
> >  
> > +_filter_xfsv4_mount_options
> > +
> >  # clear any mkfs options so that we can directly specify the options we need to
> >  # be able to test the features bitmask behaviour correctly.
> >  MKFS_OPTIONS=
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html