[PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12

[PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12

[Darrick J. Wong]

Hi all,

Here are new tests for problems that were fixed in upstream Linux
between 5.9 and 5.12.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=kernel-regressions
---
 .gitignore             |    1 
 common/filter          |   24 +++
 src/Makefile           |    4 -
 src/chprojid_fail.c    |   92 ++++++++++++
 src/deduperace.c       |  370 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/1300     |  109 ++++++++++++++
 tests/generic/1300.out |    2 
 tests/generic/947      |  118 +++++++++++++++
 tests/generic/947.out  |   15 ++
 tests/generic/948      |   92 ++++++++++++
 tests/generic/948.out  |    9 +
 tests/generic/949      |   51 +++++++
 tests/generic/949.out  |    2 
 tests/generic/group    |    4 +
 tests/xfs/050          |   30 +---
 tests/xfs/122          |    1 
 tests/xfs/122.out      |    1 
 tests/xfs/299          |   30 +---
 tests/xfs/758          |   59 ++++++++
 tests/xfs/758.out      |    2 
 tests/xfs/759          |  100 +++++++++++++
 tests/xfs/759.out      |    2 
 tests/xfs/760          |   68 +++++++++
 tests/xfs/760.out      |    9 +
 tests/xfs/761          |   45 ++++++
 tests/xfs/761.out      |    1 
 tests/xfs/765          |   71 +++++++++
 tests/xfs/765.out      |    4 +
 tests/xfs/915          |  162 +++++++++++++++++++++
 tests/xfs/915.out      |  151 ++++++++++++++++++++
 tests/xfs/group        |    6 +
 31 files changed, 1584 insertions(+), 51 deletions(-)
 create mode 100644 src/chprojid_fail.c
 create mode 100644 src/deduperace.c
 create mode 100755 tests/generic/1300
 create mode 100644 tests/generic/1300.out
 create mode 100755 tests/generic/947
 create mode 100644 tests/generic/947.out
 create mode 100755 tests/generic/948
 create mode 100644 tests/generic/948.out
 create mode 100755 tests/generic/949
 create mode 100644 tests/generic/949.out
 create mode 100755 tests/xfs/758
 create mode 100644 tests/xfs/758.out
 create mode 100755 tests/xfs/759
 create mode 100644 tests/xfs/759.out
 create mode 100755 tests/xfs/760
 create mode 100644 tests/xfs/760.out
 create mode 100755 tests/xfs/761
 create mode 100644 tests/xfs/761.out
 create mode 100755 tests/xfs/765
 create mode 100644 tests/xfs/765.out
 create mode 100755 tests/xfs/915
 create mode 100644 tests/xfs/915.out

[PATCH 01/10] xfs: test regression in xfs_bmap_validate_extent

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test to make sure that we can have realtime files
with xattr blocks and not trip the verifiers.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/758     |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/758.out |    2 ++
 tests/xfs/group   |    1 +
 3 files changed, 62 insertions(+)
 create mode 100755 tests/xfs/758
 create mode 100644 tests/xfs/758.out


diff --git a/tests/xfs/758 b/tests/xfs/758
new file mode 100755
index 00000000..a247701e
--- /dev/null
+++ b/tests/xfs/758
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 758
+#
+# This is a regression test for commit d0c20d38af13 "xfs: fix
+# xfs_bmap_validate_extent_raw when checking attr fork of rt files", which
+# fixes the bmap record validator so that it will not check the attr fork
+# extent mappings of a realtime file against the size of the realtime volume.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+
+rm -f $seqres.full
+
+# Format filesystem with very tiny realtime volume
+_scratch_mkfs -r size=256k > $seqres.full
+_scratch_mount >> $seqres.full
+
+# Create a realtime file
+$XFS_IO_PROG -f -R -c 'pwrite 0 64k' -c stat $SCRATCH_MNT/v >> $seqres.full
+
+# Add enough xattr data to force creation of xattr blocks at a higher address
+# on the data device than the size of the realtime volume
+for i in `seq 0 16`; do
+	$ATTR_PROG -s user.test$i $SCRATCH_MNT/v < $SCRATCH_MNT/v >> $seqres.full
+done
+
+# Force flushing extent maps to disk to trip the verifier
+_scratch_cycle_mount
+
+# Now let that unmount
+echo Silence is golden.
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/758.out b/tests/xfs/758.out
new file mode 100644
index 00000000..6d219f8e
--- /dev/null
+++ b/tests/xfs/758.out
@@ -0,0 +1,2 @@
+QA output created by 758
+Silence is golden.
diff --git a/tests/xfs/group b/tests/xfs/group
index 754fc7be..4dd9901f 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -500,4 +500,5 @@
 525 auto quick mkfs
 526 auto quick mkfs
 527 auto quick quota
+758 auto quick rw attr realtime
 763 auto quick rw realtime

[PATCH 02/10] generic: test reflink and copy_file_range behavior with O_SYNC and FS_XFLAG_SYNC files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add two regression tests to make sure that FICLONERANGE and the splice
based copy_file_range actually flush all data and metadata to disk
before the call ends.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/generic/947     |  118 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/947.out |   15 ++++++
 tests/generic/948     |   92 ++++++++++++++++++++++++++++++++++++++
 tests/generic/948.out |    9 ++++
 tests/generic/group   |    2 +
 5 files changed, 236 insertions(+)
 create mode 100755 tests/generic/947
 create mode 100644 tests/generic/947.out
 create mode 100755 tests/generic/948
 create mode 100644 tests/generic/948.out


diff --git a/tests/generic/947 b/tests/generic/947
new file mode 100755
index 00000000..d0edb876
--- /dev/null
+++ b/tests/generic/947
@@ -0,0 +1,118 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 947
+#
+# Make sure that reflink forces the log out if we open the file with O_SYNC or
+# set FS_XFLAG_SYNC on the file.  We test that it actually forced the log by
+# using dm-error to shut down the fs without flushing the log and then
+# remounting to check file contents.  This is a regression test for commit
+# 5ffce3cc22a0 ("xfs: force the log after remapping a synchronous-writes file")
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	_dmerror_unmount
+	_dmerror_cleanup
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+. ./common/dmerror
+
+# real QA test starts here
+_supported_fs generic
+_require_dm_target error
+_require_scratch_reflink
+_require_xfs_io_command "chattr" "s"
+_require_cp_reflink
+
+rm -f $seqres.full
+
+# Format filesystem and set up quota limits
+_scratch_mkfs > $seqres.full
+_require_metadata_journaling $SCRATCH_DEV
+_dmerror_init
+_dmerror_mount
+
+# Test that O_SYNC actually results in file data being written even if the
+# fs immediately dies
+echo "test o_sync write"
+$XFS_IO_PROG -x -f -s -c "pwrite -S 0x58 0 1m -b 1m" $SCRATCH_MNT/0 >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/0 | _filter_scratch
+
+# Set up initial files for reflink test
+$XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m -b 1m' $SCRATCH_MNT/a >> $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x59 0 1m -b 1m' $SCRATCH_MNT/c >> $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/e
+_cp_reflink $SCRATCH_MNT/c $SCRATCH_MNT/d
+touch $SCRATCH_MNT/b
+sync
+
+# Test that reflink forces dirty data/metadata to disk when destination file
+# opened with O_SYNC
+echo "test reflink flag not set o_sync"
+$XFS_IO_PROG -x -s -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# Test that reflink to a shared file forces dirty data/metadata to disk when
+# destination is opened with O_SYNC
+echo "test reflink flag already set o_sync"
+$XFS_IO_PROG -x -s -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/d >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/d | _filter_scratch
+
+# Set up the two files with chattr +S
+rm -f $SCRATCH_MNT/b $SCRATCH_MNT/d
+_cp_reflink $SCRATCH_MNT/c $SCRATCH_MNT/d
+touch $SCRATCH_MNT/b
+chattr +S $SCRATCH_MNT/b $SCRATCH_MNT/d
+sync
+
+# Test that reflink forces dirty data/metadata to disk when destination file
+# has the sync iflag set
+echo "test reflink flag not set iflag"
+$XFS_IO_PROG -x -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# Test that reflink to a shared file forces dirty data/metadata to disk when
+# destination file has the sync iflag set
+echo "test reflink flag already set iflag"
+$XFS_IO_PROG -x -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/d >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/d | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/947.out b/tests/generic/947.out
new file mode 100644
index 00000000..05ba10d1
--- /dev/null
+++ b/tests/generic/947.out
@@ -0,0 +1,15 @@
+QA output created by 947
+test o_sync write
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/0
+test reflink flag not set o_sync
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/b
+test reflink flag already set o_sync
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/d
+test reflink flag not set iflag
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/b
+test reflink flag already set iflag
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/d
diff --git a/tests/generic/948 b/tests/generic/948
new file mode 100755
index 00000000..b79cd279
--- /dev/null
+++ b/tests/generic/948
@@ -0,0 +1,92 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 948
+#
+# Make sure that copy_file_range forces the log out if we open the file with
+# O_SYNC or set FS_XFLAG_SYNC on the file.  We test that it actually forced the
+# log by using dm-error to shut down the fs without flushing the log and then
+# remounting to check file contents.  This is a regression test for commit
+# 5ffce3cc22a0 ("xfs: force the log after remapping a synchronous-writes file")
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	_dmerror_unmount
+	_dmerror_cleanup
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmerror
+
+# real QA test starts here
+_supported_fs generic
+_require_dm_target error
+_require_xfs_io_command "chattr" "s"
+_require_scratch
+
+rm -f $seqres.full
+
+# Format filesystem and set up quota limits
+_scratch_mkfs > $seqres.full
+_require_metadata_journaling $SCRATCH_DEV
+_dmerror_init
+_dmerror_mount
+
+# Test that O_SYNC actually results in file data being written even if the
+# fs immediately dies
+echo "test o_sync write"
+$XFS_IO_PROG -x -f -s -c "pwrite -S 0x58 0 1m -b 1m" $SCRATCH_MNT/0 >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/0 | _filter_scratch
+
+# Set up initial files for copy test
+$XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m -b 1m' $SCRATCH_MNT/a >> $seqres.full
+touch $SCRATCH_MNT/b
+sync
+
+# Test that unaligned copy file range forces dirty data/metadata to disk when
+# destination file opened with O_SYNC
+echo "test unaligned copy range o_sync"
+$XFS_IO_PROG -x -s -c "copy_range -s 13 -d 13 -l 1048550 $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# Set up dest file with chattr +S
+rm -f $SCRATCH_MNT/b
+touch $SCRATCH_MNT/b
+chattr +S $SCRATCH_MNT/b
+sync
+
+# Test that unaligned copy file range forces dirty data/metadata to disk when
+# destination file has the sync iflag set
+echo "test unaligned copy range iflag"
+$XFS_IO_PROG -x -c "copy_range -s 13 -d 13 -l 1048550 $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/948.out b/tests/generic/948.out
new file mode 100644
index 00000000..eec6c0dc
--- /dev/null
+++ b/tests/generic/948.out
@@ -0,0 +1,9 @@
+QA output created by 948
+test o_sync write
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/0
+test unaligned copy range o_sync
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/a
+2a715d2093b5aca82783a0c5943ac0b8  SCRATCH_MNT/b
+test unaligned copy range iflag
+310f146ce52077fcd3308dcbe7632bb2  SCRATCH_MNT/a
+2a715d2093b5aca82783a0c5943ac0b8  SCRATCH_MNT/b
diff --git a/tests/generic/group b/tests/generic/group
index 84db3789..d5cfdd51 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -628,3 +628,5 @@
 623 auto quick shutdown
 624 auto quick verity
 625 auto quick verity
+947 auto quick rw clone
+948 auto quick rw copy_range

[PATCH 03/10] xfs: test rtalloc alignment and math errors

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add a couple of regression tests for "xfs: make sure the rt allocator
doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
finsert operations are aligned to rt extent size".

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/759     |  100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/759.out |    2 +
 tests/xfs/760     |   68 ++++++++++++++++++++++++++++++++++++
 tests/xfs/760.out |    9 +++++
 tests/xfs/group   |    2 +
 5 files changed, 181 insertions(+)
 create mode 100755 tests/xfs/759
 create mode 100644 tests/xfs/759.out
 create mode 100755 tests/xfs/760
 create mode 100644 tests/xfs/760.out


diff --git a/tests/xfs/759 b/tests/xfs/759
new file mode 100755
index 00000000..8558fe30
--- /dev/null
+++ b/tests/xfs/759
@@ -0,0 +1,100 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 759
+#
+# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
+# allocator doesn't run off the end") which fixes an overflow error in the
+# _near realtime allocator.  If the rt bitmap ends exactly at the end of a
+# block and the number of rt extents is large enough to allow an allocation
+# request larger than the maximum extent size, it's possible that during a
+# large allocation request, the allocator will fail to constrain maxlen on the
+# second run through the loop, and the rt bitmap range check will run right off
+# the end of the rtbitmap file.  When this happens, xfs triggers a verifier
+# error and returns EFSCORRUPTED.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+_require_test_program "punch-alternating"
+
+rm -f $seqres.full
+
+# Format filesystem to get the block size
+_scratch_mkfs > $seqres.full
+_scratch_mount >> $seqres.full
+
+blksz=$(_get_block_size $SCRATCH_MNT)
+rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
+rextblks=$((rextsize / blksz))
+
+echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
+
+_scratch_unmount
+
+# Format filesystem with a realtime volume whose size fits the following:
+# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
+# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes.
+
+rtsize1=$((2097151 * blksz))
+rtsize2=$((8 * blksz * rextsize))
+rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 ))
+
+echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full
+
+test $rtsize -gt $rtsize1 || \
+	_notrun "scratch rt device too small, need $rtsize1 bytes"
+test $rtsize -gt $rtsize2 || \
+	_notrun "scratch rt device too small, need $rtsize2 bytes"
+
+rtsize=$((rtsize - (rtsize % rtsize2)))
+
+echo "rt size will be $rtsize" >> $seqres.full
+
+_scratch_mkfs -r size=$rtsize >> $seqres.full
+_scratch_mount >> $seqres.full
+
+# Make sure the root directory has rtinherit set so our test file will too
+$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+
+# Allocate some stuff at the start, to force the first falloc of the ouch file
+# to happen somewhere in the middle of the rt volume
+$XFS_IO_PROG -f -c 'falloc 0 64m' "$SCRATCH_MNT/b"
+$here/src/punch-alternating -i $((rextblks * 2)) -s $((rextblks)) "$SCRATCH_MNT/b"
+
+avail="$(df -P "$SCRATCH_MNT" | awk 'END {print $4}')"1
+toobig="$((avail * 2))"
+
+# falloc the ouch file in the middle of the rt extent to exercise the near
+# allocator in the last step.
+$XFS_IO_PROG -f -c 'falloc 0 1g' "$SCRATCH_MNT/ouch"
+
+# Try to get the near allocator to overflow on an allocation that matches
+# exactly one of the rtsummary size levels.  This should return ENOSPC and
+# not EFSCORRUPTED.
+$XFS_IO_PROG -f -c "falloc 0 ${toobig}k" "$SCRATCH_MNT/ouch"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/759.out b/tests/xfs/759.out
new file mode 100644
index 00000000..df693d50
--- /dev/null
+++ b/tests/xfs/759.out
@@ -0,0 +1,2 @@
+QA output created by 759
+fallocate: No space left on device
diff --git a/tests/xfs/760 b/tests/xfs/760
new file mode 100755
index 00000000..7f2b52d4
--- /dev/null
+++ b/tests/xfs/760
@@ -0,0 +1,68 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 760
+#
+# Make sure we validate realtime extent size alignment for fallocate modes.
+# This is a regression test for fe341eb151ec ("xfs: ensure that fpunch,
+# fcollapse, and finsert operations are aligned to rt extent size")
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+_require_xfs_io_command "fcollapse"
+_require_xfs_io_command "finsert"
+_require_xfs_io_command "funshare"
+_require_xfs_io_command "fzero"
+_require_xfs_io_command "falloc"
+
+rm -f $seqres.full
+
+# Format filesystem with a 256k realtime extent size
+_scratch_mkfs -r extsize=256k > $seqres.full
+_scratch_mount >> $seqres.full
+
+blksz=$(_get_block_size $SCRATCH_MNT)
+rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
+rextblks=$((rextsize / blksz))
+
+echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
+
+# Make sure the root directory has rtinherit set so our test file will too
+$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+
+sz=$((rextsize * 100))
+range="$((blksz * 3)) $blksz"
+
+for verb in fpunch finsert fcollapse fzero funshare falloc; do
+	echo "test $verb"
+	$XFS_IO_PROG -f -c "falloc 0 $sz" "$SCRATCH_MNT/b"
+	$XFS_IO_PROG -f -c "$verb $range" "$SCRATCH_MNT/b"
+	rm -f "$SCRATCH_MNT/b"
+	_scratch_cycle_mount
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/760.out b/tests/xfs/760.out
new file mode 100644
index 00000000..3d73c6fa
--- /dev/null
+++ b/tests/xfs/760.out
@@ -0,0 +1,9 @@
+QA output created by 760
+test fpunch
+test finsert
+fallocate: Invalid argument
+test fcollapse
+fallocate: Invalid argument
+test fzero
+test funshare
+test falloc
diff --git a/tests/xfs/group b/tests/xfs/group
index 4dd9901f..318468b5 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -501,4 +501,6 @@
 526 auto quick mkfs
 527 auto quick quota
 758 auto quick rw attr realtime
+759 auto quick rw realtime
+760 auto quick rw collapse punch insert zero prealloc
 763 auto quick rw realtime

[PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In "mkfs: set required parts of the realtime geometry before computing
log geometry" we made sure that mkfs set up enough of the fs geometry to
compute the minimum xfs log size correctly when formatting the
filesystem.  This is the regression test for that issue.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/761     |   45 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/761.out |    1 +
 tests/xfs/group   |    1 +
 3 files changed, 47 insertions(+)
 create mode 100755 tests/xfs/761
 create mode 100644 tests/xfs/761.out


diff --git a/tests/xfs/761 b/tests/xfs/761
new file mode 100755
index 00000000..b9770d90
--- /dev/null
+++ b/tests/xfs/761
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 761
+#
+# Make sure mkfs sets up enough of the rt geometry that we can compute the
+# correct min log size for formatting the fs.
+#
+# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
+# required parts of the realtime geometry before computing log geometry").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+
+rm -f $seqres.full
+
+# Format a tiny filesystem to force minimum log size, then see if it mounts
+_scratch_mkfs -r size=1m -d size=100m > $seqres.full
+_scratch_mount >> $seqres.full
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/761.out b/tests/xfs/761.out
new file mode 100644
index 00000000..8c9d9e90
--- /dev/null
+++ b/tests/xfs/761.out
@@ -0,0 +1 @@
+QA output created by 761
diff --git a/tests/xfs/group b/tests/xfs/group
index 318468b5..87badd56 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -503,4 +503,5 @@
 758 auto quick rw attr realtime
 759 auto quick rw realtime
 760 auto quick rw collapse punch insert zero prealloc
+761 auto quick realtime
 763 auto quick rw realtime

[PATCH 05/10] common/filter: refactor quota report filtering

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

xfs/299 and xfs/050 share the same function to filter quota reporting
into a format suitable for the golden output.  Refactor this so that we
can use it in a new test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/filter |   24 ++++++++++++++++++++++++
 tests/xfs/050 |   30 ++++++------------------------
 tests/xfs/299 |   30 ++++++------------------------
 3 files changed, 36 insertions(+), 48 deletions(-)


diff --git a/common/filter b/common/filter
index 2f3277f7..2efbbd99 100644
--- a/common/filter
+++ b/common/filter
@@ -637,5 +637,29 @@ _filter_getcap()
         sed -e "s/= //" -e "s/\+/=/g"
 }
 
+# Filter user/group/project id numbers out of quota reports, and standardize
+# the block counts to use filesystem block size.  Callers must set the id and
+# bsize variables before calling this function.
+_filter_quota_report()
+{
+	test -n "$id" || echo "id must be set"
+	test -n "$bsize" || echo "block size must be set"
+
+	tr -s '[:space:]' | \
+	perl -npe '
+		s/^\#'$id' /[NAME] /g;
+		s/^\#0 \d+ /[ROOT] 0 /g;
+		s/6 days/7 days/g' |
+	perl -npe '
+		$val = 0;
+		if ($ENV{'LARGE_SCRATCH_DEV'}) {
+			$val = $ENV{'NUM_SPACE_FILES'};
+		}
+		s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
+	sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
+	perl -npe '
+		s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
+}
+
 # make sure this script returns success
 /bin/true
diff --git a/tests/xfs/050 b/tests/xfs/050
index 53412a13..1df97537 100755
--- a/tests/xfs/050
+++ b/tests/xfs/050
@@ -47,24 +47,6 @@ bhard=$(( 1000 * $bsize ))
 isoft=4
 ihard=10
 
-_filter_report()
-{
-	tr -s '[:space:]' | \
-	perl -npe '
-		s/^\#'$id' /[NAME] /g;
-		s/^\#0 \d+ /[ROOT] 0 /g;
-		s/6 days/7 days/g' |
-	perl -npe '
-		$val = 0;
-		if ($ENV{'LARGE_SCRATCH_DEV'}) {
-			$val = $ENV{'NUM_SPACE_FILES'};
-		}
-		s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
-	sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
-	perl -npe '
-		s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
-}
-
 # The actual point at which limit enforcement takes place for the
 # hard block limit is variable depending on filesystem blocksize,
 # and iosize.  What we want to test is that the limit is enforced
@@ -84,7 +66,7 @@ _filter_and_check_blks()
 			}
 			s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
 		}
-	' | _filter_report
+	' | _filter_quota_report
 }
 
 _qsetup()
@@ -134,7 +116,7 @@ _exercise()
 	echo "*** report no quota settings" | tee -a $seqres.full
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	echo "*** report initial settings" | tee -a $seqres.full
@@ -147,7 +129,7 @@ _exercise()
 		$SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	echo "*** push past the soft inode limit" | tee -a $seqres.full
@@ -159,7 +141,7 @@ _exercise()
 	$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	echo "*** push past the soft block limit" | tee -a $seqres.full
@@ -169,7 +151,7 @@ _exercise()
 		-c "warn -b -$type 0 $id" $SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	# Note: for quota accounting (not enforcement), EDQUOT is not expected
@@ -183,7 +165,7 @@ _exercise()
 		-c "warn -i -$type 0 $id" $SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	# Note: for quota accounting (not enforcement), EDQUOT is not expected
diff --git a/tests/xfs/299 b/tests/xfs/299
index 15e0edf6..b862e67e 100755
--- a/tests/xfs/299
+++ b/tests/xfs/299
@@ -40,24 +40,6 @@ _require_xfs_quota
 _require_xfs_mkfs_crc
 _require_xfs_crc
 
-_filter_report()
-{
-	tr -s '[:space:]' | \
-	perl -npe '
-		s/^\#'$id' /[NAME] /g;
-		s/^\#0 \d+ /[ROOT] 0 /g;
-		s/6 days/7 days/g' |
-	perl -npe '
-		$val = 0;
-		if ($ENV{'LARGE_SCRATCH_DEV'}) {
-			$val = $ENV{'NUM_SPACE_FILES'};
-		}
-		s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
-	sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
-	perl -npe '
-		s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
-}
-
 # The actual point at which limit enforcement takes place for the
 # hard block limit is variable depending on filesystem blocksize,
 # and iosize.  What we want to test is that the limit is enforced
@@ -77,7 +59,7 @@ _filter_and_check_blks()
 			}
 			s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
 		}
-	' | _filter_report
+	' | _filter_quota_report
 }
 
 _qsetup()
@@ -120,7 +102,7 @@ _exercise()
 	echo "*** report no quota settings" | tee -a $seqres.full
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	echo "*** report initial settings" | tee -a $seqres.full
@@ -133,7 +115,7 @@ _exercise()
 		$SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	echo "*** push past the soft inode limit" | tee -a $seqres.full
@@ -145,7 +127,7 @@ _exercise()
 	$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	echo "*** push past the soft block limit" | tee -a $seqres.full
@@ -155,7 +137,7 @@ _exercise()
 		-c "warn -b -$type 0 $id" $SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	# Note: for quota accounting (not enforcement), EDQUOT is not expected
@@ -169,7 +151,7 @@ _exercise()
 		-c "warn -i -$type 0 $id" $SCRATCH_DEV
 	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
 		-c "repquota -birnN -$type" $SCRATCH_DEV |
-		_filter_report | LC_COLLATE=POSIX sort -ru
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
 
 	echo
 	# Note: for quota accounting (not enforcement), EDQUOT is not expected

[PATCH 06/10] xfs: test quota softlimit warning functionality

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Make sure that quota softlimits work, which is to say that one can
exceed the softlimit up to warnlimit times before it starts enforcing
that.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/915     |  162 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/915.out |  151 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/group   |    1 
 3 files changed, 314 insertions(+)
 create mode 100755 tests/xfs/915
 create mode 100644 tests/xfs/915.out


diff --git a/tests/xfs/915 b/tests/xfs/915
new file mode 100755
index 00000000..a2cdbbb7
--- /dev/null
+++ b/tests/xfs/915
@@ -0,0 +1,162 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 915
+#
+# Check that quota softlimit warnings work the way they should.  This means
+# that we can disobey the softlimit up to warnlimit times before it turns into
+# hard(er) enforcement.  This is a functional test for quota warnings, but
+# since the functionality has been broken for decades, this is also a
+# regression test for commit 4b8628d57b72 ("xfs: actually bump warning counts
+# when we send warnings").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/quota
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_quota
+_require_scratch
+
+rm -f $seqres.full
+
+qsetup()
+{
+	opt=$1
+	enforce=0
+	if [ $opt = "u" -o $opt = "uno" ]; then
+		type=u
+		eval `_choose_uid`
+	elif [ $opt = "g" -o $opt = "gno" ]; then
+		type=g
+		eval `_choose_gid`
+	elif [ $opt = "p" -o $opt = "pno" ]; then
+		type=p
+		eval `_choose_prid`
+	fi
+	[ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
+
+	echo "Using type=$type id=$id" >> $seqres.full
+}
+
+exercise() {
+	_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
+	cat $tmp.mkfs >>$seqres.full
+
+	# keep the blocksize and data size for dd later
+	. $tmp.mkfs
+
+	_qmount
+
+	qsetup $1
+
+	echo "Using type=$type id=$id" >>$seqres.full
+
+	echo
+	echo "*** report initial settings" | tee -a $seqres.full
+	$XFS_QUOTA_PROG -x \
+		-c "limit -$type isoft=3 ihard=500000 $id" \
+		-c "warn -$type -i -d 13" \
+		$SCRATCH_DEV
+	$XFS_QUOTA_PROG -x \
+		-c "state -$type" >> $seqres.full
+	$XFS_QUOTA_PROG -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** push past the soft inode limit" | tee -a $seqres.full
+	_file_as_id $SCRATCH_MNT/softok1 $id $type $bsize 0
+	_file_as_id $SCRATCH_MNT/softok2 $id $type $bsize 0
+	_file_as_id $SCRATCH_MNT/softok3 $id $type $bsize 0
+	_file_as_id $SCRATCH_MNT/softwarn1 $id $type $bsize 0
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** push further past the soft inode limit" | tee -a $seqres.full
+	for warn_nr in $(seq 2 5); do
+		_file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
+	done
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** push past the soft inode warning limit" | tee -a $seqres.full
+	for warn_nr in $(seq 6 15); do
+		_file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
+	done
+	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+		-c "repquota -birnN -$type" $SCRATCH_DEV |
+		_filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+	echo
+	echo "*** unmount"
+	_scratch_unmount
+}
+
+_scratch_mkfs > $seqres.full
+_scratch_mount >> $seqres.full
+
+chmod a+rwx $SCRATCH_MNT $seqres.full	# arbitrary users will write here
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+_scratch_unmount
+
+cat >$tmp.projects <<EOF
+1:$SCRATCH_MNT
+EOF
+
+cat >$tmp.projid <<EOF
+root:0
+scratch:1
+EOF
+
+projid_file="$tmp.projid"
+
+echo "*** user"
+_qmount_option "uquota"
+exercise u
+
+echo "*** group"
+_qmount_option "gquota"
+exercise g
+
+echo "*** uqnoenforce"
+_qmount_option "uqnoenforce"
+exercise uno
+
+echo "*** gqnoenforce"
+_qmount_option "gqnoenforce"
+exercise gno
+
+echo "*** pquota"
+_qmount_option "pquota"
+exercise p
+
+echo "*** pqnoenforce"
+_qmount_option "pqnoenforce"
+exercise pno
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/915.out b/tests/xfs/915.out
new file mode 100644
index 00000000..c3bb855e
--- /dev/null
+++ b/tests/xfs/915.out
@@ -0,0 +1,151 @@
+QA output created by 915
+*** user
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
+
+*** unmount
+*** group
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
+
+*** unmount
+*** uqnoenforce
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** unmount
+*** gqnoenforce
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** unmount
+*** pquota
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 02 [7 days] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 06 [7 days] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 15 3 500000 13 [7 days] 0 0 0 00 [--------]
+
+*** unmount
+*** pqnoenforce
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data     = bsize=XXX blocks=XXX, imaxpct=PCT
+         = sunit=XXX swidth=XXX, unwritten=X
+naming   =VERN bsize=XXX
+log      =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** unmount
diff --git a/tests/xfs/group b/tests/xfs/group
index 87badd56..d7aafc94 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -505,3 +505,4 @@
 760 auto quick rw collapse punch insert zero prealloc
 761 auto quick realtime
 763 auto quick rw realtime
+915 auto quick quota

[PATCH 07/10] xfs/122: fix test for xfs_attr_shortform_t conversion

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The typedef xfs_attr_shortform_t was converted to a struct in 5.10.
Update this test to pass.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
---
 tests/xfs/122     |    1 -
 tests/xfs/122.out |    1 +
 2 files changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/xfs/122 b/tests/xfs/122
index a4248031..322e1d81 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -72,7 +72,6 @@ for hdr in /usr/include/xfs/xfs*.h; do
 done
 
 # missing:
-# xfs_attr_shortform_t
 # xfs_trans_header_t
 
 cat >$tmp.ignore <<EOF
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 45c42e59..cfe09c6d 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -62,6 +62,7 @@ sizeof(struct xfs_agfl) = 36
 sizeof(struct xfs_attr3_leaf_hdr) = 80
 sizeof(struct xfs_attr3_leafblock) = 88
 sizeof(struct xfs_attr3_rmt_hdr) = 56
+sizeof(struct xfs_attr_shortform) = 8
 sizeof(struct xfs_btree_block) = 72
 sizeof(struct xfs_btree_block_lhdr) = 64
 sizeof(struct xfs_btree_block_shdr) = 48

[PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Create a test to make sure that dedupe actually locks the file ranges
correctly before starting the content comparison and keeps them locked
until the operation completes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 src/Makefile          |    2 
 src/deduperace.c      |  370 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/949     |   51 +++++++
 tests/generic/949.out |    2 
 tests/generic/group   |    1 
 5 files changed, 425 insertions(+), 1 deletion(-)
 create mode 100644 src/deduperace.c
 create mode 100755 tests/generic/949
 create mode 100644 tests/generic/949.out


diff --git a/src/Makefile b/src/Makefile
index 811b24e4..38ee6718 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -21,7 +21,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \
 
 LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \
-	locktest unwritten_mmap bulkstat_unlink_test \
+	locktest unwritten_mmap bulkstat_unlink_test deduperace \
 	bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
 	stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
 	seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \
diff --git a/src/deduperace.c b/src/deduperace.c
new file mode 100644
index 00000000..b252d436
--- /dev/null
+++ b/src/deduperace.c
@@ -0,0 +1,370 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2021 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <djwong@kernel.org>
+ *
+ * Race pwrite/mwrite with dedupe to see if we got the locking right.
+ *
+ * File writes and mmap writes should not be able to change the src_fd's
+ * contents after dedupe prep has verified that the file contents are the same.
+ */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <sys/ioctl.h>
+#include <linux/fs.h>
+#include <string.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdlib.h>
+#include <errno.h>
+
+#define GOOD_BYTE		0x58
+#define BAD_BYTE		0x66
+
+#ifndef FIDEDUPERANGE
+/* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */
+#define FILE_DEDUPE_RANGE_SAME		0
+#define FILE_DEDUPE_RANGE_DIFFERS	1
+
+/* from struct btrfs_ioctl_file_extent_same_info */
+struct file_dedupe_range_info {
+	__s64 dest_fd;		/* in - destination file */
+	__u64 dest_offset;	/* in - start of extent in destination */
+	__u64 bytes_deduped;	/* out - total # of bytes we were able
+				 * to dedupe from this file. */
+	/* status of this dedupe operation:
+	 * < 0 for error
+	 * == FILE_DEDUPE_RANGE_SAME if dedupe succeeds
+	 * == FILE_DEDUPE_RANGE_DIFFERS if data differs
+	 */
+	__s32 status;		/* out - see above description */
+	__u32 reserved;		/* must be zero */
+};
+
+/* from struct btrfs_ioctl_file_extent_same_args */
+struct file_dedupe_range {
+	__u64 src_offset;	/* in - start of extent in source */
+	__u64 src_length;	/* in - length of extent */
+	__u16 dest_count;	/* in - total elements in info array */
+	__u16 reserved1;	/* must be zero */
+	__u32 reserved2;	/* must be zero */
+	struct file_dedupe_range_info info[0];
+};
+#define FIDEDUPERANGE	_IOWR(0x94, 54, struct file_dedupe_range)
+#endif /* FIDEDUPERANGE */
+
+static int fd1, fd2;
+static loff_t offset = 37; /* Nice low offset to trick the compare */
+static loff_t blksz;
+
+/* Continuously dirty the pagecache for the region being dupe-tested. */
+void *
+mwriter(
+	void		*data)
+{
+	volatile char	*p;
+
+	p = mmap(NULL, blksz, PROT_WRITE, MAP_SHARED, fd1, 0);
+	if (p == MAP_FAILED) {
+		perror("mmap");
+		exit(2);
+	}
+
+	while (1) {
+		*(p + offset) = BAD_BYTE;
+		*(p + offset) = GOOD_BYTE;
+	}
+}
+
+/* Continuously write to the region being dupe-tested. */
+void *
+pwriter(
+	void		*data)
+{
+	char		v;
+	ssize_t		sz;
+
+	while (1) {
+		v = BAD_BYTE;
+		sz = pwrite(fd1, &v, sizeof(v), offset);
+		if (sz != sizeof(v)) {
+			perror("pwrite0");
+			exit(2);
+		}
+
+		v = GOOD_BYTE;
+		sz = pwrite(fd1, &v, sizeof(v), offset);
+		if (sz != sizeof(v)) {
+			perror("pwrite1");
+			exit(2);
+		}
+	}
+
+	return NULL;
+}
+
+static inline void
+complain(
+	loff_t	offset,
+	char	bad)
+{
+	fprintf(stderr, "ASSERT: offset %llu should be 0x%x, got 0x%x!\n",
+			(unsigned long long)offset, GOOD_BYTE, bad);
+	abort();
+}
+
+/* Make sure the destination file pagecache never changes. */
+void *
+mreader(
+	void		*data)
+{
+	volatile char	*p;
+
+	p = mmap(NULL, blksz, PROT_READ, MAP_SHARED, fd2, 0);
+	if (p == MAP_FAILED) {
+		perror("mmap");
+		exit(2);
+	}
+
+	while (1) {
+		if (*(p + offset) != GOOD_BYTE)
+			complain(offset, *(p + offset));
+	}
+}
+
+/* Make sure the destination file never changes. */
+void *
+preader(
+	void		*data)
+{
+	char		v;
+	ssize_t		sz;
+
+	while (1) {
+		sz = pread(fd2, &v, sizeof(v), offset);
+		if (sz != sizeof(v)) {
+			perror("pwrite0");
+			exit(2);
+		}
+
+		if (v != GOOD_BYTE)
+			complain(offset, v);
+	}
+
+	return NULL;
+}
+
+void
+print_help(const char *progname)
+{
+	printf("Usage: %s [-b blksz] [-c dir] [-n nr_ops] [-o offset] [-r] [-w] [-v]\n",
+			progname);
+	printf("-b sets the block size (default is autoconfigured)\n");
+	printf("-c chdir to this path before starting\n");
+	printf("-n controls the number of dedupe ops (default 10000)\n");
+	printf("-o reads and writes to this offset (default 37)\n");
+	printf("-r uses pread instead of mmap read.\n");
+	printf("-v prints status updates.\n");
+	printf("-w uses pwrite instead of mmap write.\n");
+}
+
+int
+main(
+	int		argc,
+	char		*argv[])
+{
+	struct file_dedupe_range *fdr;
+	char		*Xbuf;
+	void		*(*reader_fn)(void *) = mreader;
+	void		*(*writer_fn)(void *) = mwriter;
+	unsigned long	same = 0;
+	unsigned long	differs = 0;
+	unsigned long	i, nr_ops = 10000;
+	ssize_t		sz;
+	pthread_t	reader, writer;
+	int		verbose = 0;
+	int		c;
+	int		ret;
+
+	while ((c = getopt(argc, argv, "b:c:n:o:rvw")) != -1) {
+		switch (c) {
+		case 'b':
+			errno = 0;
+			blksz = strtoul(optarg, NULL, 0);
+			if (errno) {
+				perror(optarg);
+				exit(1);
+			}
+			break;
+		case 'c':
+			ret = chdir(optarg);
+			if (ret) {
+				perror("chdir");
+				exit(1);
+			}
+			break;
+		case 'n':
+			errno = 0;
+			nr_ops = strtoul(optarg, NULL, 0);
+			if (errno) {
+				perror(optarg);
+				exit(1);
+			}
+			break;
+		case 'o':
+			errno = 0;
+			offset = strtoul(optarg, NULL, 0);
+			if (errno) {
+				perror(optarg);
+				exit(1);
+			}
+			break;
+		case 'r':
+			reader_fn = preader;
+			break;
+		case 'v':
+			verbose = 1;
+			break;
+		case 'w':
+			writer_fn = pwriter;
+			break;
+		default:
+			print_help(argv[0]);
+			exit(1);
+			break;
+		}
+	}
+
+	fdr = malloc(sizeof(struct file_dedupe_range) +
+			sizeof(struct file_dedupe_range_info));
+	if (!fdr) {
+		perror("malloc");
+		exit(1);
+	}
+
+	/* Initialize both files. */
+	fd1 = open("file1", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
+	if (fd1 < 0) {
+		perror("file1");
+		exit(1);
+	}
+
+	fd2 = open("file2", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
+	if (fd2 < 0) {
+		perror("file2");
+		exit(1);
+	}
+
+	if (blksz <= 0) {
+		struct stat	statbuf;
+
+		ret = fstat(fd1, &statbuf);
+		if (ret) {
+			perror("file1 stat");
+			exit(1);
+		}
+		blksz = statbuf.st_blksize;
+	}
+
+	if (offset >= blksz) {
+		fprintf(stderr, "offset (%llu) < blksz (%llu)?\n",
+				(unsigned long long)offset,
+				(unsigned long long)blksz);
+		exit(1);
+	}
+
+	Xbuf = malloc(blksz);
+	if (!Xbuf) {
+		perror("malloc buffer");
+		exit(1);
+	}
+	memset(Xbuf, GOOD_BYTE, blksz);
+
+	sz = pwrite(fd1, Xbuf, blksz, 0);
+	if (sz != blksz) {
+		perror("file1 write");
+		exit(1);
+	}
+
+	sz = pwrite(fd2, Xbuf, blksz, 0);
+	if (sz != blksz) {
+		perror("file2 write");
+		exit(1);
+	}
+
+	ret = fsync(fd1);
+	if (ret) {
+		perror("file1 fsync");
+		exit(1);
+	}
+
+	ret = fsync(fd2);
+	if (ret) {
+		perror("file2 fsync");
+		exit(1);
+	}
+
+	/* Start our reader and writer threads. */
+	ret = pthread_create(&reader, NULL, reader_fn, NULL);
+	if (ret) {
+		fprintf(stderr, "rthread: %s\n", strerror(ret));
+		exit(1);
+	}
+
+	ret = pthread_create(&writer, NULL, writer_fn, NULL);
+	if (ret) {
+		fprintf(stderr, "wthread: %s\n", strerror(ret));
+		exit(1);
+	}
+
+	/*
+	 * Now start deduping.  If the contents match, fd1's blocks will be
+	 * remapped into fd2, which is why the writer thread targets fd1 and
+	 * the reader checks fd2 to make sure that none of fd1's writes ever
+	 * make it into fd2.
+	 */
+	for (i = 1; i <= nr_ops; i++) {
+		fdr->src_offset = 0;
+		fdr->src_length = blksz;
+		fdr->dest_count = 1;
+		fdr->reserved1 = 0;
+		fdr->reserved2 = 0;
+		fdr->info[0].dest_fd = fd2;
+		fdr->info[0].dest_offset = 0;
+		fdr->info[0].reserved = 0;
+
+		ret = ioctl(fd1, FIDEDUPERANGE, fdr);
+		if (ret) {
+			perror("dedupe");
+			exit(2);
+		}
+
+		switch (fdr->info[0].status) {
+		case FILE_DEDUPE_RANGE_DIFFERS:
+			differs++;
+			break;
+		case FILE_DEDUPE_RANGE_SAME:
+			same++;
+			break;
+		default:
+			fprintf(stderr, "deduperange: %s\n",
+					strerror(-fdr->info[0].status));
+			exit(2);
+			break;
+		}
+
+		if (verbose && (i % 337) == 0)
+			printf("nr_ops: %lu; same: %lu; differs: %lu\n",
+					i, same, differs);
+	}
+
+	if (verbose)
+		printf("nr_ops: %lu; same: %lu; differs: %lu\n", i - 1, same,
+				differs);
+
+	/* Program termination will kill the threads and close the files. */
+	return 0;
+}
diff --git a/tests/generic/949 b/tests/generic/949
new file mode 100755
index 00000000..3951490b
--- /dev/null
+++ b/tests/generic/949
@@ -0,0 +1,51 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 949
+#
+# Make sure that mmap and file writers racing with FIDEDUPERANGE cannot write
+# to the file after the dedupe prep function has decided that the file contents
+# are identical and we can therefore go ahead with the remapping.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch_dedupe
+
+rm -f $seqres.full
+
+nr_ops=$((TIME_FACTOR * 10000))
+
+# Format filesystem
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Test once with mmap writes
+$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops
+
+# Test again with pwrites for the lulz
+$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops -w
+
+echo Silence is golden.
+# success, all done
+status=0
+exit
diff --git a/tests/generic/949.out b/tests/generic/949.out
new file mode 100644
index 00000000..2998b46c
--- /dev/null
+++ b/tests/generic/949.out
@@ -0,0 +1,2 @@
+QA output created by 949
+Silence is golden.
diff --git a/tests/generic/group b/tests/generic/group
index d5cfdd51..778aa8c4 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -630,3 +630,4 @@
 625 auto quick verity
 947 auto quick rw clone
 948 auto quick rw copy_range
+949 auto quick rw dedupe clone

[PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

wenli xie reported a buffer cache deadlock when an overlayfs is mounted
atop xfs and overlayfs tries to replace a single-nlink file with a
whiteout file.  This test reproduces that deadlock.

Reported-by: wenli xie <wlxie7296@gmail.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/1300     |  109 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/1300.out |    2 +
 tests/generic/group    |    1 
 3 files changed, 112 insertions(+)
 create mode 100755 tests/generic/1300
 create mode 100644 tests/generic/1300.out


diff --git a/tests/generic/1300 b/tests/generic/1300
new file mode 100755
index 00000000..10df44e3
--- /dev/null
+++ b/tests/generic/1300
@@ -0,0 +1,109 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1300
+#
+# Reproducer for a deadlock in xfs_rename reported by Wenli Xie.
+#
+# When overlayfs is running on top of xfs and the user unlinks a file in the
+# overlay, overlayfs will create a whiteout inode and ask us to "rename" the
+# whiteout file atop the one being unlinked.  If the file being unlinked loses
+# its one nlink, we then have to put the inode on the unlinked list.
+#
+# This requires us to grab the AGI buffer of the whiteout inode to take it
+# off the unlinked list (which is where whiteouts are created) and to grab
+# the AGI buffer of the file being deleted.  If the whiteout was created in
+# a higher numbered AG than the file being deleted, we'll lock the AGIs in
+# the wrong order and deadlock.
+#
+# Note that this test doesn't do anything xfs-specific so it's a generic test.
+# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock
+# in xfs_rename").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	stop_workers
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs."
+
+modprobe -q overlay
+grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs."
+
+rm -f $seqres.full
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+
+mkdir $SCRATCH_MNT/lowerdir
+mkdir $SCRATCH_MNT/lowerdir1
+mkdir $SCRATCH_MNT/lowerdir/etc
+mkdir $SCRATCH_MNT/workers
+echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf
+touch $SCRATCH_MNT/running
+
+stop_workers() {
+	test -e $SCRATCH_MNT/running || return
+	rm -f $SCRATCH_MNT/running
+
+	while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do
+		wait
+	done
+}
+
+worker() {
+	local tag="$1"
+	local mergedir="$SCRATCH_MNT/merged$tag"
+	local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1"
+	local u="upperdir=$SCRATCH_MNT/upperdir$tag"
+	local w="workdir=$SCRATCH_MNT/workdir$tag"
+	local i="index=off,nfs_export=off"
+
+	touch $SCRATCH_MNT/workers/$tag
+	while test -e $SCRATCH_MNT/running; do
+		rm -rf $SCRATCH_MNT/merged$tag
+		rm -rf $SCRATCH_MNT/upperdir$tag
+		rm -rf $SCRATCH_MNT/workdir$tag
+		mkdir $SCRATCH_MNT/merged$tag
+		mkdir $SCRATCH_MNT/workdir$tag
+		mkdir $SCRATCH_MNT/upperdir$tag
+
+		mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir
+		mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
+		touch $mergedir/etc/access.conf
+		mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
+		touch $mergedir/etc/access.conf
+		umount $mergedir
+	done
+	rm -f $SCRATCH_MNT/workers/$tag
+}
+
+for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do
+	worker $i &
+done
+
+sleep $((30 * TIME_FACTOR))
+stop_workers
+
+echo Silence is golden.
+# success, all done
+status=0
+exit
diff --git a/tests/generic/1300.out b/tests/generic/1300.out
new file mode 100644
index 00000000..5805d30d
--- /dev/null
+++ b/tests/generic/1300.out
@@ -0,0 +1,2 @@
+QA output created by 1300
+Silence is golden.
diff --git a/tests/generic/group b/tests/generic/group
index 778aa8c4..2233a59d 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -631,3 +631,4 @@
 947 auto quick rw clone
 948 auto quick rw copy_range
 949 auto quick rw dedupe clone
+1300 auto rw overlay

[PATCH 10/10] xfs: test delalloc quota leak when chprojid fails

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This is a regression test for a bug in the XFS implementation of
FSSETXATTR.  When we try to change a file's project id, the quota
reservation code will update the incore quota reservations for delayed
allocation blocks.  Unfortunately, it does this before we finish
validating all the FSSETXATTR parameters, which means that if we decide
to bail out, we also fail to undo the incore changes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 .gitignore          |    1 +
 src/Makefile        |    2 +
 src/chprojid_fail.c |   92 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/765       |   71 +++++++++++++++++++++++++++++++++++++++
 tests/xfs/765.out   |    4 ++
 tests/xfs/group     |    1 +
 6 files changed, 170 insertions(+), 1 deletion(-)
 create mode 100644 src/chprojid_fail.c
 create mode 100755 tests/xfs/765
 create mode 100644 tests/xfs/765.out


diff --git a/.gitignore b/.gitignore
index 03c03be5..3af8e207 100644
--- a/.gitignore
+++ b/.gitignore
@@ -58,6 +58,7 @@
 /src/bulkstat_null_ocount
 /src/bulkstat_unlink_test
 /src/bulkstat_unlink_test_modified
+/src/chprojid_fail
 /src/cloner
 /src/dbtest
 /src/devzero
diff --git a/src/Makefile b/src/Makefile
index 38ee6718..3d729a34 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -29,7 +29,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 	attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
 	dio-invalidate-cache stat_test t_encrypted_d_revalidate \
 	attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
-	fscrypt-crypt-util bulkstat_null_ocount splice-test
+	fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail
 
 SUBDIRS = log-writes perf
 
diff --git a/src/chprojid_fail.c b/src/chprojid_fail.c
new file mode 100644
index 00000000..8c5b5fee
--- /dev/null
+++ b/src/chprojid_fail.c
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2021 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <djwong@kernel.org>
+ *
+ * Regression test for failing to undo delalloc quota reservations when
+ * changing project id and we fail some other FSSETXATTR validation.
+ */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <linux/fs.h>
+
+static char zerobuf[65536];
+
+int
+main(
+	int		argc,
+	char		*argv[])
+{
+	struct fsxattr	fa;
+	ssize_t		sz;
+	int		fd, ret;
+
+	if (argc < 2) {
+		printf("Usage: %s filename\n", argv[0]);
+		return 1;
+	}
+
+	fd = open(argv[1], O_CREAT | O_TRUNC | O_RDWR, 0600);
+	if (fd < 0) {
+		perror(argv[1]);
+		return 2;
+	}
+
+	/* Zero the project id and the extent size hint. */
+	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
+	if (ret) {
+		perror("FSGETXATTR check file");
+		return 2;
+	}
+
+	if (fa.fsx_projid != 0 || fa.fsx_extsize != 0) {
+		fa.fsx_projid = 0;
+		fa.fsx_extsize = 0;
+		ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
+		if (ret) {
+			perror("FSSETXATTR zeroing");
+			return 2;
+		}
+	}
+
+	/* Dirty a few kb of a file to create delalloc extents. */
+	sz = write(fd, zerobuf, sizeof(zerobuf));
+	if (sz != sizeof(zerobuf)) {
+		perror("delalloc write");
+		return 2;
+	}
+
+	/*
+	 * The regression we're trying to test happens when the fsxattr input
+	 * validation decides to bail out after the chown quota reservation has
+	 * been made on a file containing delalloc extents.  Extent size hints
+	 * can't be set on non-empty files and we can't check the value until
+	 * we've reserved resources and taken the file's ILOCK, so this is a
+	 * perfect vector for triggering this condition.  In this way we set up
+	 * a FSSETXATTR call that will fail.
+	 */
+	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
+	if (ret) {
+		perror("FSGETXATTR");
+		return 2;
+	}
+
+	fa.fsx_projid = 23652;
+	fa.fsx_extsize = 2;
+	fa.fsx_xflags |= FS_XFLAG_EXTSIZE;
+
+	ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
+	if (ret) {
+		printf("FSSETXATTRR should fail: %s\n", strerror(errno));
+		return 0;
+	}
+
+	/* Uhoh, that FSSETXATTR call should have failed! */
+	return 3;
+}
diff --git a/tests/xfs/765 b/tests/xfs/765
new file mode 100755
index 00000000..68b89ce3
--- /dev/null
+++ b/tests/xfs/765
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 765
+#
+# Regression test for failing to undo delalloc quota reservations when changing
+# project id but we fail some other part of FSSETXATTR validation.  If we fail
+# the test, we trip debugging assertions in dmesg.  This is a regression test
+# for commit 1aecf3734a95 ("xfs: fix chown leaking delalloc quota blocks when
+# fssetxattr fails").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/quota
+
+# real QA test starts here
+_supported_fs xfs
+_require_command "$FILEFRAG_PROG" filefrag
+_require_test_program "chprojid_fail"
+_require_quota
+_require_scratch
+
+rm -f $seqres.full
+
+echo "Format filesystem" | tee -a $seqres.full
+_scratch_mkfs > $seqres.full
+_qmount_option 'prjquota'
+_qmount
+_require_prjquota $SCRATCH_DEV
+
+# Make sure that a regular buffered write produces delalloc reservations.
+$XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null
+$FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \
+	_notrun "test requires delayed allocation writes"
+rm -f $SCRATCH_MNT/testy
+
+echo "Run test program"
+$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
+$here/src/chprojid_fail $SCRATCH_MNT/blah
+
+# The regression we're testing for is an accounting bug involving delalloc
+# reservations.  FSSETXATTR does not itself cause dirty data writeback, so we
+# assume that if the file still has delalloc extents, then it must have had
+# them when chprojid_fail was running, and therefore the test was set up
+# correctly.  There's a slight chance that background writeback can sneak in
+# and flush the file, but this should be a small enough gap.
+$FILEFRAG_PROG -v $SCRATCH_MNT/blah 2>&1 | grep -q delalloc || \
+	echo "file didn't get delalloc extents, test invalid?"
+
+# Make a note of current quota status for diagnostic purposes
+$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/765.out b/tests/xfs/765.out
new file mode 100644
index 00000000..d5f8fc11
--- /dev/null
+++ b/tests/xfs/765.out
@@ -0,0 +1,4 @@
+QA output created by 765
+Format filesystem
+Run test program
+FSSETXATTRR should fail: Invalid argument
diff --git a/tests/xfs/group b/tests/xfs/group
index d7aafc94..84468d10 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -505,4 +505,5 @@
 760 auto quick rw collapse punch insert zero prealloc
 761 auto quick realtime
 763 auto quick rw realtime
+765 auto quick quota
 915 auto quick quota

Re: [PATCH 03/10] xfs: test rtalloc alignment and math errors

[Chandan Babu R]

On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Add a couple of regression tests for "xfs: make sure the rt allocator
> doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
> finsert operations are aligned to rt extent size".
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/759     |  100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/759.out |    2 +
>  tests/xfs/760     |   68 ++++++++++++++++++++++++++++++++++++
>  tests/xfs/760.out |    9 +++++
>  tests/xfs/group   |    2 +
>  5 files changed, 181 insertions(+)
>  create mode 100755 tests/xfs/759
>  create mode 100644 tests/xfs/759.out
>  create mode 100755 tests/xfs/760
>  create mode 100644 tests/xfs/760.out
>
>
> diff --git a/tests/xfs/759 b/tests/xfs/759
> new file mode 100755
> index 00000000..8558fe30
> --- /dev/null
> +++ b/tests/xfs/759
> @@ -0,0 +1,100 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 759
> +#
> +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
> +# allocator doesn't run off the end") which fixes an overflow error in the
> +# _near realtime allocator.  If the rt bitmap ends exactly at the end of a
> +# block and the number of rt extents is large enough to allow an allocation
> +# request larger than the maximum extent size, it's possible that during a
> +# large allocation request, the allocator will fail to constrain maxlen on the
> +# second run through the loop, and the rt bitmap range check will run right off
> +# the end of the rtbitmap file.  When this happens, xfs triggers a verifier
> +# error and returns EFSCORRUPTED.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_realtime
> +_require_test_program "punch-alternating"
> +
> +rm -f $seqres.full
> +
> +# Format filesystem to get the block size
> +_scratch_mkfs > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +blksz=$(_get_block_size $SCRATCH_MNT)
> +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
> +rextblks=$((rextsize / blksz))
> +
> +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
> +
> +_scratch_unmount
> +
> +# Format filesystem with a realtime volume whose size fits the following:
> +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.

Shouldn't the multiplier be RT extent size rather than FS block size?

> +# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes.

i.e The bits in one rt bitmap block map (NBBY * blksz * rextsize) bytes of an
rt device. Hence to have the bitmap end at a fs block boundary the
corresponding rt device size should be a multiple of this product. Is my
understanding correct?

> +
> +rtsize1=$((2097151 * blksz))
> +rtsize2=$((8 * blksz * rextsize))
> +rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 ))
> +
> +echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full
> +
> +test $rtsize -gt $rtsize1 || \
> +	_notrun "scratch rt device too small, need $rtsize1 bytes"
> +test $rtsize -gt $rtsize2 || \
> +	_notrun "scratch rt device too small, need $rtsize2 bytes"
> +
> +rtsize=$((rtsize - (rtsize % rtsize2)))
> +

--
chandan

Re: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes

[Chandan Babu R]

On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> In "mkfs: set required parts of the realtime geometry before computing
> log geometry" we made sure that mkfs set up enough of the fs geometry to
> compute the minimum xfs log size correctly when formatting the
> filesystem.  This is the regression test for that issue.
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/761     |   45 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/761.out |    1 +
>  tests/xfs/group   |    1 +
>  3 files changed, 47 insertions(+)
>  create mode 100755 tests/xfs/761
>  create mode 100644 tests/xfs/761.out
>
>
> diff --git a/tests/xfs/761 b/tests/xfs/761
> new file mode 100755
> index 00000000..b9770d90
> --- /dev/null
> +++ b/tests/xfs/761
> @@ -0,0 +1,45 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 761
> +#
> +# Make sure mkfs sets up enough of the rt geometry that we can compute the
> +# correct min log size for formatting the fs.
> +#
> +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
> +# required parts of the realtime geometry before computing log geometry").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_realtime
> +
> +rm -f $seqres.full
> +
> +# Format a tiny filesystem to force minimum log size, then see if it mounts
> +_scratch_mkfs -r size=1m -d size=100m > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/761.out b/tests/xfs/761.out
> new file mode 100644
> index 00000000..8c9d9e90
> --- /dev/null
> +++ b/tests/xfs/761.out
> @@ -0,0 +1 @@
> +QA output created by 761
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 318468b5..87badd56 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -503,4 +503,5 @@
>  758 auto quick rw attr realtime
>  759 auto quick rw realtime
>  760 auto quick rw collapse punch insert zero prealloc
> +761 auto quick realtime

Maybe "mount" should be added to the above list. Rest looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

-- 
chandan

Re: [PATCH 05/10] common/filter: refactor quota report filtering

[Chandan Babu R]

On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> xfs/299 and xfs/050 share the same function to filter quota reporting
> into a format suitable for the golden output.  Refactor this so that we
> can use it in a new test.
>

Looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/filter |   24 ++++++++++++++++++++++++
>  tests/xfs/050 |   30 ++++++------------------------
>  tests/xfs/299 |   30 ++++++------------------------
>  3 files changed, 36 insertions(+), 48 deletions(-)
>
>
> diff --git a/common/filter b/common/filter
> index 2f3277f7..2efbbd99 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -637,5 +637,29 @@ _filter_getcap()
>          sed -e "s/= //" -e "s/\+/=/g"
>  }
>  
> +# Filter user/group/project id numbers out of quota reports, and standardize
> +# the block counts to use filesystem block size.  Callers must set the id and
> +# bsize variables before calling this function.
> +_filter_quota_report()
> +{
> +	test -n "$id" || echo "id must be set"
> +	test -n "$bsize" || echo "block size must be set"
> +
> +	tr -s '[:space:]' | \
> +	perl -npe '
> +		s/^\#'$id' /[NAME] /g;
> +		s/^\#0 \d+ /[ROOT] 0 /g;
> +		s/6 days/7 days/g' |
> +	perl -npe '
> +		$val = 0;
> +		if ($ENV{'LARGE_SCRATCH_DEV'}) {
> +			$val = $ENV{'NUM_SPACE_FILES'};
> +		}
> +		s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
> +	sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
> +	perl -npe '
> +		s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
> +}
> +
>  # make sure this script returns success
>  /bin/true
> diff --git a/tests/xfs/050 b/tests/xfs/050
> index 53412a13..1df97537 100755
> --- a/tests/xfs/050
> +++ b/tests/xfs/050
> @@ -47,24 +47,6 @@ bhard=$(( 1000 * $bsize ))
>  isoft=4
>  ihard=10
>  
> -_filter_report()
> -{
> -	tr -s '[:space:]' | \
> -	perl -npe '
> -		s/^\#'$id' /[NAME] /g;
> -		s/^\#0 \d+ /[ROOT] 0 /g;
> -		s/6 days/7 days/g' |
> -	perl -npe '
> -		$val = 0;
> -		if ($ENV{'LARGE_SCRATCH_DEV'}) {
> -			$val = $ENV{'NUM_SPACE_FILES'};
> -		}
> -		s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
> -	sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
> -	perl -npe '
> -		s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
> -}
> -
>  # The actual point at which limit enforcement takes place for the
>  # hard block limit is variable depending on filesystem blocksize,
>  # and iosize.  What we want to test is that the limit is enforced
> @@ -84,7 +66,7 @@ _filter_and_check_blks()
>  			}
>  			s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
>  		}
> -	' | _filter_report
> +	' | _filter_quota_report
>  }
>  
>  _qsetup()
> @@ -134,7 +116,7 @@ _exercise()
>  	echo "*** report no quota settings" | tee -a $seqres.full
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	echo "*** report initial settings" | tee -a $seqres.full
> @@ -147,7 +129,7 @@ _exercise()
>  		$SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	echo "*** push past the soft inode limit" | tee -a $seqres.full
> @@ -159,7 +141,7 @@ _exercise()
>  	$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	echo "*** push past the soft block limit" | tee -a $seqres.full
> @@ -169,7 +151,7 @@ _exercise()
>  		-c "warn -b -$type 0 $id" $SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	# Note: for quota accounting (not enforcement), EDQUOT is not expected
> @@ -183,7 +165,7 @@ _exercise()
>  		-c "warn -i -$type 0 $id" $SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	# Note: for quota accounting (not enforcement), EDQUOT is not expected
> diff --git a/tests/xfs/299 b/tests/xfs/299
> index 15e0edf6..b862e67e 100755
> --- a/tests/xfs/299
> +++ b/tests/xfs/299
> @@ -40,24 +40,6 @@ _require_xfs_quota
>  _require_xfs_mkfs_crc
>  _require_xfs_crc
>  
> -_filter_report()
> -{
> -	tr -s '[:space:]' | \
> -	perl -npe '
> -		s/^\#'$id' /[NAME] /g;
> -		s/^\#0 \d+ /[ROOT] 0 /g;
> -		s/6 days/7 days/g' |
> -	perl -npe '
> -		$val = 0;
> -		if ($ENV{'LARGE_SCRATCH_DEV'}) {
> -			$val = $ENV{'NUM_SPACE_FILES'};
> -		}
> -		s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
> -	sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
> -	perl -npe '
> -		s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
> -}
> -
>  # The actual point at which limit enforcement takes place for the
>  # hard block limit is variable depending on filesystem blocksize,
>  # and iosize.  What we want to test is that the limit is enforced
> @@ -77,7 +59,7 @@ _filter_and_check_blks()
>  			}
>  			s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
>  		}
> -	' | _filter_report
> +	' | _filter_quota_report
>  }
>  
>  _qsetup()
> @@ -120,7 +102,7 @@ _exercise()
>  	echo "*** report no quota settings" | tee -a $seqres.full
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	echo "*** report initial settings" | tee -a $seqres.full
> @@ -133,7 +115,7 @@ _exercise()
>  		$SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	echo "*** push past the soft inode limit" | tee -a $seqres.full
> @@ -145,7 +127,7 @@ _exercise()
>  	$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	echo "*** push past the soft block limit" | tee -a $seqres.full
> @@ -155,7 +137,7 @@ _exercise()
>  		-c "warn -b -$type 0 $id" $SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	# Note: for quota accounting (not enforcement), EDQUOT is not expected
> @@ -169,7 +151,7 @@ _exercise()
>  		-c "warn -i -$type 0 $id" $SCRATCH_DEV
>  	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
>  		-c "repquota -birnN -$type" $SCRATCH_DEV |
> -		_filter_report | LC_COLLATE=POSIX sort -ru
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
>  
>  	echo
>  	# Note: for quota accounting (not enforcement), EDQUOT is not expected


-- 
chandan

Re: [PATCH 06/10] xfs: test quota softlimit warning functionality

[Chandan Babu R]

On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Make sure that quota softlimits work, which is to say that one can
> exceed the softlimit up to warnlimit times before it starts enforcing
> that.
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/915     |  162 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/915.out |  151 +++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/group   |    1
>  3 files changed, 314 insertions(+)
>  create mode 100755 tests/xfs/915
>  create mode 100644 tests/xfs/915.out
>
>
> diff --git a/tests/xfs/915 b/tests/xfs/915
> new file mode 100755
> index 00000000..a2cdbbb7
> --- /dev/null
> +++ b/tests/xfs/915
> @@ -0,0 +1,162 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 915
> +#
> +# Check that quota softlimit warnings work the way they should.  This means
> +# that we can disobey the softlimit up to warnlimit times before it turns into
> +# hard(er) enforcement.  This is a functional test for quota warnings, but
> +# since the functionality has been broken for decades, this is also a
> +# regression test for commit 4b8628d57b72 ("xfs: actually bump warning counts
> +# when we send warnings").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/quota
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_xfs_quota
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +qsetup()
> +{
> +	opt=$1
> +	enforce=0
> +	if [ $opt = "u" -o $opt = "uno" ]; then
> +		type=u
> +		eval `_choose_uid`
> +	elif [ $opt = "g" -o $opt = "gno" ]; then
> +		type=g
> +		eval `_choose_gid`
> +	elif [ $opt = "p" -o $opt = "pno" ]; then
> +		type=p
> +		eval `_choose_prid`
> +	fi
> +	[ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
> +
> +	echo "Using type=$type id=$id" >> $seqres.full
> +}
> +
> +exercise() {
> +	_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
> +	cat $tmp.mkfs >>$seqres.full
> +
> +	# keep the blocksize and data size for dd later
> +	. $tmp.mkfs
> +
> +	_qmount
> +
> +	qsetup $1
> +
> +	echo "Using type=$type id=$id" >>$seqres.full
> +
> +	echo
> +	echo "*** report initial settings" | tee -a $seqres.full
> +	$XFS_QUOTA_PROG -x \
> +		-c "limit -$type isoft=3 ihard=500000 $id" \
> +		-c "warn -$type -i -d 13" \
> +		$SCRATCH_DEV
> +	$XFS_QUOTA_PROG -x \
> +		-c "state -$type" >> $seqres.full
> +	$XFS_QUOTA_PROG -x \
> +		-c "repquota -birnN -$type" $SCRATCH_DEV |
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> +	echo
> +	echo "*** push past the soft inode limit" | tee -a $seqres.full
> +	_file_as_id $SCRATCH_MNT/softok1 $id $type $bsize 0
> +	_file_as_id $SCRATCH_MNT/softok2 $id $type $bsize 0
> +	_file_as_id $SCRATCH_MNT/softok3 $id $type $bsize 0
> +	_file_as_id $SCRATCH_MNT/softwarn1 $id $type $bsize 0
> +	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> +		-c "repquota -birnN -$type" $SCRATCH_DEV |
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> +	echo
> +	echo "*** push further past the soft inode limit" | tee -a $seqres.full
> +	for warn_nr in $(seq 2 5); do
> +		_file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
> +	done
> +	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> +		-c "repquota -birnN -$type" $SCRATCH_DEV |
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> +	echo
> +	echo "*** push past the soft inode warning limit" | tee -a $seqres.full
> +	for warn_nr in $(seq 6 15); do
> +		_file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
> +	done
> +	$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> +		-c "repquota -birnN -$type" $SCRATCH_DEV |
> +		_filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> +	echo
> +	echo "*** unmount"
> +	_scratch_unmount
> +}
> +
> +_scratch_mkfs > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +chmod a+rwx $SCRATCH_MNT $seqres.full	# arbitrary users will write here
> +bsize=$(_get_file_block_size $SCRATCH_MNT)
> +_scratch_unmount
> +
> +cat >$tmp.projects <<EOF
> +1:$SCRATCH_MNT
> +EOF
> +
> +cat >$tmp.projid <<EOF
> +root:0
> +scratch:1
> +EOF
> +
> +projid_file="$tmp.projid"
> +
> +echo "*** user"
> +_qmount_option "uquota"
> +exercise u
> +
> +echo "*** group"
> +_qmount_option "gquota"
> +exercise g
> +
> +echo "*** uqnoenforce"
> +_qmount_option "uqnoenforce"
> +exercise uno
> +
> +echo "*** gqnoenforce"
> +_qmount_option "gqnoenforce"
> +exercise gno
> +
> +echo "*** pquota"
> +_qmount_option "pquota"
> +exercise p
> +
> +echo "*** pqnoenforce"
> +_qmount_option "pqnoenforce"
> +exercise pno
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/915.out b/tests/xfs/915.out
> new file mode 100644
> index 00000000..c3bb855e
> --- /dev/null
> +++ b/tests/xfs/915.out
> @@ -0,0 +1,151 @@
> +QA output created by 915
> +*** user
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data     = bsize=XXX blocks=XXX, imaxpct=PCT
> +         = sunit=XXX swidth=XXX, unwritten=X
> +naming   =VERN bsize=XXX
> +log      =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** group
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data     = bsize=XXX blocks=XXX, imaxpct=PCT
> +         = sunit=XXX swidth=XXX, unwritten=X
> +naming   =VERN bsize=XXX
> +log      =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** uqnoenforce
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data     = bsize=XXX blocks=XXX, imaxpct=PCT
> +         = sunit=XXX swidth=XXX, unwritten=X
> +naming   =VERN bsize=XXX
> +log      =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** gqnoenforce
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data     = bsize=XXX blocks=XXX, imaxpct=PCT
> +         = sunit=XXX swidth=XXX, unwritten=X
> +naming   =VERN bsize=XXX
> +log      =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** pquota
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data     = bsize=XXX blocks=XXX, imaxpct=PCT
> +         = sunit=XXX swidth=XXX, unwritten=X
> +naming   =VERN bsize=XXX
> +log      =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 02 [7 days] 0 0 0 00 [--------]

At this point in the test we have created 4 files.
1. softok{1,2,3}
2. softwarn1

So we have exceeded the soft inode limit (i.e. 3) once. But the warning has
been issued twice.

_file_as_id() changes the project id of parent of each of the above files.  In
this case all the above listed files have $SCRATCH_MNT as the parent. So by
the time softok2 is created we have already reached the soft inode limit of 3
(parent and the two softok{1,2} files) and creation of softok3 and softwarn1
generates the two warnings listed above. If this explaination is correct,
shouldn't 'Used' inode count have a value of 5 (including the inode associated
with $SCRATCH_MNT)?

--
chandan

Re: [PATCH 07/10] xfs/122: fix test for xfs_attr_shortform_t conversion

[Christoph Hellwig]

On Mon, Mar 08, 2021 at 08:40:42PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> The typedef xfs_attr_shortform_t was converted to a struct in 5.10.
> Update this test to pass.

Or rather the typedef was removed, the struct always existed.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 06/10] xfs: test quota softlimit warning functionality

[Chandan Babu R]

On 11 Mar 2021 at 16:40, Chandan Babu R wrote:
> On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
>> From: Darrick J. Wong <djwong@kernel.org>
>>
>> Make sure that quota softlimits work, which is to say that one can
>> exceed the softlimit up to warnlimit times before it starts enforcing
>> that.
>>
>> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>> ---
>>  tests/xfs/915     |  162 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  tests/xfs/915.out |  151 +++++++++++++++++++++++++++++++++++++++++++++++++
>>  tests/xfs/group   |    1
>>  3 files changed, 314 insertions(+)
>>  create mode 100755 tests/xfs/915
>>  create mode 100644 tests/xfs/915.out
>>
>>
>> diff --git a/tests/xfs/915 b/tests/xfs/915

[...]

>> +*** report initial settings
>> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
>> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
>> +
>> +*** push past the soft inode limit
>> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
>> +[NAME] 0 0 0 00 [--------] 4 3 500000 02 [7 days] 0 0 0 00 [--------]
>
> At this point in the test we have created 4 files.
> 1. softok{1,2,3}
> 2. softwarn1
>
> So we have exceeded the soft inode limit (i.e. 3) once. But the warning has
> been issued twice.
>
> _file_as_id() changes the project id of parent of each of the above files.  In
> this case all the above listed files have $SCRATCH_MNT as the parent. So by
> the time softok2 is created we have already reached the soft inode limit of 3
> (parent and the two softok{1,2} files) and creation of softok3 and softwarn1
> generates the two warnings listed above. If this explaination is correct,
> shouldn't 'Used' inode count have a value of 5 (including the inode associated
> with $SCRATCH_MNT)?

Sorry, I missed the fact that _file_as_id() reverts back the project id of
$SCRATCH_MNT just before returning.

The patch looks good.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

-- 
chandan

Re: [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE

[Chandan Babu R]

On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Create a test to make sure that dedupe actually locks the file ranges
> correctly before starting the content comparison and keeps them locked
> until the operation completes.
>

Looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  src/Makefile          |    2 
>  src/deduperace.c      |  370 +++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/949     |   51 +++++++
>  tests/generic/949.out |    2 
>  tests/generic/group   |    1 
>  5 files changed, 425 insertions(+), 1 deletion(-)
>  create mode 100644 src/deduperace.c
>  create mode 100755 tests/generic/949
>  create mode 100644 tests/generic/949.out
>
>
> diff --git a/src/Makefile b/src/Makefile
> index 811b24e4..38ee6718 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -21,7 +21,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \
>  
>  LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
>  	preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \
> -	locktest unwritten_mmap bulkstat_unlink_test \
> +	locktest unwritten_mmap bulkstat_unlink_test deduperace \
>  	bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
>  	stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
>  	seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \
> diff --git a/src/deduperace.c b/src/deduperace.c
> new file mode 100644
> index 00000000..b252d436
> --- /dev/null
> +++ b/src/deduperace.c
> @@ -0,0 +1,370 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2021 Oracle.  All Rights Reserved.
> + * Author: Darrick J. Wong <djwong@kernel.org>
> + *
> + * Race pwrite/mwrite with dedupe to see if we got the locking right.
> + *
> + * File writes and mmap writes should not be able to change the src_fd's
> + * contents after dedupe prep has verified that the file contents are the same.
> + */
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <sys/mman.h>
> +#include <sys/ioctl.h>
> +#include <linux/fs.h>
> +#include <string.h>
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +#include <pthread.h>
> +#include <stdlib.h>
> +#include <errno.h>
> +
> +#define GOOD_BYTE		0x58
> +#define BAD_BYTE		0x66
> +
> +#ifndef FIDEDUPERANGE
> +/* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */
> +#define FILE_DEDUPE_RANGE_SAME		0
> +#define FILE_DEDUPE_RANGE_DIFFERS	1
> +
> +/* from struct btrfs_ioctl_file_extent_same_info */
> +struct file_dedupe_range_info {
> +	__s64 dest_fd;		/* in - destination file */
> +	__u64 dest_offset;	/* in - start of extent in destination */
> +	__u64 bytes_deduped;	/* out - total # of bytes we were able
> +				 * to dedupe from this file. */
> +	/* status of this dedupe operation:
> +	 * < 0 for error
> +	 * == FILE_DEDUPE_RANGE_SAME if dedupe succeeds
> +	 * == FILE_DEDUPE_RANGE_DIFFERS if data differs
> +	 */
> +	__s32 status;		/* out - see above description */
> +	__u32 reserved;		/* must be zero */
> +};
> +
> +/* from struct btrfs_ioctl_file_extent_same_args */
> +struct file_dedupe_range {
> +	__u64 src_offset;	/* in - start of extent in source */
> +	__u64 src_length;	/* in - length of extent */
> +	__u16 dest_count;	/* in - total elements in info array */
> +	__u16 reserved1;	/* must be zero */
> +	__u32 reserved2;	/* must be zero */
> +	struct file_dedupe_range_info info[0];
> +};
> +#define FIDEDUPERANGE	_IOWR(0x94, 54, struct file_dedupe_range)
> +#endif /* FIDEDUPERANGE */
> +
> +static int fd1, fd2;
> +static loff_t offset = 37; /* Nice low offset to trick the compare */
> +static loff_t blksz;
> +
> +/* Continuously dirty the pagecache for the region being dupe-tested. */
> +void *
> +mwriter(
> +	void		*data)
> +{
> +	volatile char	*p;
> +
> +	p = mmap(NULL, blksz, PROT_WRITE, MAP_SHARED, fd1, 0);
> +	if (p == MAP_FAILED) {
> +		perror("mmap");
> +		exit(2);
> +	}
> +
> +	while (1) {
> +		*(p + offset) = BAD_BYTE;
> +		*(p + offset) = GOOD_BYTE;
> +	}
> +}
> +
> +/* Continuously write to the region being dupe-tested. */
> +void *
> +pwriter(
> +	void		*data)
> +{
> +	char		v;
> +	ssize_t		sz;
> +
> +	while (1) {
> +		v = BAD_BYTE;
> +		sz = pwrite(fd1, &v, sizeof(v), offset);
> +		if (sz != sizeof(v)) {
> +			perror("pwrite0");
> +			exit(2);
> +		}
> +
> +		v = GOOD_BYTE;
> +		sz = pwrite(fd1, &v, sizeof(v), offset);
> +		if (sz != sizeof(v)) {
> +			perror("pwrite1");
> +			exit(2);
> +		}
> +	}
> +
> +	return NULL;
> +}
> +
> +static inline void
> +complain(
> +	loff_t	offset,
> +	char	bad)
> +{
> +	fprintf(stderr, "ASSERT: offset %llu should be 0x%x, got 0x%x!\n",
> +			(unsigned long long)offset, GOOD_BYTE, bad);
> +	abort();
> +}
> +
> +/* Make sure the destination file pagecache never changes. */
> +void *
> +mreader(
> +	void		*data)
> +{
> +	volatile char	*p;
> +
> +	p = mmap(NULL, blksz, PROT_READ, MAP_SHARED, fd2, 0);
> +	if (p == MAP_FAILED) {
> +		perror("mmap");
> +		exit(2);
> +	}
> +
> +	while (1) {
> +		if (*(p + offset) != GOOD_BYTE)
> +			complain(offset, *(p + offset));
> +	}
> +}
> +
> +/* Make sure the destination file never changes. */
> +void *
> +preader(
> +	void		*data)
> +{
> +	char		v;
> +	ssize_t		sz;
> +
> +	while (1) {
> +		sz = pread(fd2, &v, sizeof(v), offset);
> +		if (sz != sizeof(v)) {
> +			perror("pwrite0");
> +			exit(2);
> +		}
> +
> +		if (v != GOOD_BYTE)
> +			complain(offset, v);
> +	}
> +
> +	return NULL;
> +}
> +
> +void
> +print_help(const char *progname)
> +{
> +	printf("Usage: %s [-b blksz] [-c dir] [-n nr_ops] [-o offset] [-r] [-w] [-v]\n",
> +			progname);
> +	printf("-b sets the block size (default is autoconfigured)\n");
> +	printf("-c chdir to this path before starting\n");
> +	printf("-n controls the number of dedupe ops (default 10000)\n");
> +	printf("-o reads and writes to this offset (default 37)\n");
> +	printf("-r uses pread instead of mmap read.\n");
> +	printf("-v prints status updates.\n");
> +	printf("-w uses pwrite instead of mmap write.\n");
> +}
> +
> +int
> +main(
> +	int		argc,
> +	char		*argv[])
> +{
> +	struct file_dedupe_range *fdr;
> +	char		*Xbuf;
> +	void		*(*reader_fn)(void *) = mreader;
> +	void		*(*writer_fn)(void *) = mwriter;
> +	unsigned long	same = 0;
> +	unsigned long	differs = 0;
> +	unsigned long	i, nr_ops = 10000;
> +	ssize_t		sz;
> +	pthread_t	reader, writer;
> +	int		verbose = 0;
> +	int		c;
> +	int		ret;
> +
> +	while ((c = getopt(argc, argv, "b:c:n:o:rvw")) != -1) {
> +		switch (c) {
> +		case 'b':
> +			errno = 0;
> +			blksz = strtoul(optarg, NULL, 0);
> +			if (errno) {
> +				perror(optarg);
> +				exit(1);
> +			}
> +			break;
> +		case 'c':
> +			ret = chdir(optarg);
> +			if (ret) {
> +				perror("chdir");
> +				exit(1);
> +			}
> +			break;
> +		case 'n':
> +			errno = 0;
> +			nr_ops = strtoul(optarg, NULL, 0);
> +			if (errno) {
> +				perror(optarg);
> +				exit(1);
> +			}
> +			break;
> +		case 'o':
> +			errno = 0;
> +			offset = strtoul(optarg, NULL, 0);
> +			if (errno) {
> +				perror(optarg);
> +				exit(1);
> +			}
> +			break;
> +		case 'r':
> +			reader_fn = preader;
> +			break;
> +		case 'v':
> +			verbose = 1;
> +			break;
> +		case 'w':
> +			writer_fn = pwriter;
> +			break;
> +		default:
> +			print_help(argv[0]);
> +			exit(1);
> +			break;
> +		}
> +	}
> +
> +	fdr = malloc(sizeof(struct file_dedupe_range) +
> +			sizeof(struct file_dedupe_range_info));
> +	if (!fdr) {
> +		perror("malloc");
> +		exit(1);
> +	}
> +
> +	/* Initialize both files. */
> +	fd1 = open("file1", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
> +	if (fd1 < 0) {
> +		perror("file1");
> +		exit(1);
> +	}
> +
> +	fd2 = open("file2", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
> +	if (fd2 < 0) {
> +		perror("file2");
> +		exit(1);
> +	}
> +
> +	if (blksz <= 0) {
> +		struct stat	statbuf;
> +
> +		ret = fstat(fd1, &statbuf);
> +		if (ret) {
> +			perror("file1 stat");
> +			exit(1);
> +		}
> +		blksz = statbuf.st_blksize;
> +	}
> +
> +	if (offset >= blksz) {
> +		fprintf(stderr, "offset (%llu) < blksz (%llu)?\n",
> +				(unsigned long long)offset,
> +				(unsigned long long)blksz);
> +		exit(1);
> +	}
> +
> +	Xbuf = malloc(blksz);
> +	if (!Xbuf) {
> +		perror("malloc buffer");
> +		exit(1);
> +	}
> +	memset(Xbuf, GOOD_BYTE, blksz);
> +
> +	sz = pwrite(fd1, Xbuf, blksz, 0);
> +	if (sz != blksz) {
> +		perror("file1 write");
> +		exit(1);
> +	}
> +
> +	sz = pwrite(fd2, Xbuf, blksz, 0);
> +	if (sz != blksz) {
> +		perror("file2 write");
> +		exit(1);
> +	}
> +
> +	ret = fsync(fd1);
> +	if (ret) {
> +		perror("file1 fsync");
> +		exit(1);
> +	}
> +
> +	ret = fsync(fd2);
> +	if (ret) {
> +		perror("file2 fsync");
> +		exit(1);
> +	}
> +
> +	/* Start our reader and writer threads. */
> +	ret = pthread_create(&reader, NULL, reader_fn, NULL);
> +	if (ret) {
> +		fprintf(stderr, "rthread: %s\n", strerror(ret));
> +		exit(1);
> +	}
> +
> +	ret = pthread_create(&writer, NULL, writer_fn, NULL);
> +	if (ret) {
> +		fprintf(stderr, "wthread: %s\n", strerror(ret));
> +		exit(1);
> +	}
> +
> +	/*
> +	 * Now start deduping.  If the contents match, fd1's blocks will be
> +	 * remapped into fd2, which is why the writer thread targets fd1 and
> +	 * the reader checks fd2 to make sure that none of fd1's writes ever
> +	 * make it into fd2.
> +	 */
> +	for (i = 1; i <= nr_ops; i++) {
> +		fdr->src_offset = 0;
> +		fdr->src_length = blksz;
> +		fdr->dest_count = 1;
> +		fdr->reserved1 = 0;
> +		fdr->reserved2 = 0;
> +		fdr->info[0].dest_fd = fd2;
> +		fdr->info[0].dest_offset = 0;
> +		fdr->info[0].reserved = 0;
> +
> +		ret = ioctl(fd1, FIDEDUPERANGE, fdr);
> +		if (ret) {
> +			perror("dedupe");
> +			exit(2);
> +		}
> +
> +		switch (fdr->info[0].status) {
> +		case FILE_DEDUPE_RANGE_DIFFERS:
> +			differs++;
> +			break;
> +		case FILE_DEDUPE_RANGE_SAME:
> +			same++;
> +			break;
> +		default:
> +			fprintf(stderr, "deduperange: %s\n",
> +					strerror(-fdr->info[0].status));
> +			exit(2);
> +			break;
> +		}
> +
> +		if (verbose && (i % 337) == 0)
> +			printf("nr_ops: %lu; same: %lu; differs: %lu\n",
> +					i, same, differs);
> +	}
> +
> +	if (verbose)
> +		printf("nr_ops: %lu; same: %lu; differs: %lu\n", i - 1, same,
> +				differs);
> +
> +	/* Program termination will kill the threads and close the files. */
> +	return 0;
> +}
> diff --git a/tests/generic/949 b/tests/generic/949
> new file mode 100755
> index 00000000..3951490b
> --- /dev/null
> +++ b/tests/generic/949
> @@ -0,0 +1,51 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 949
> +#
> +# Make sure that mmap and file writers racing with FIDEDUPERANGE cannot write
> +# to the file after the dedupe prep function has decided that the file contents
> +# are identical and we can therefore go ahead with the remapping.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/reflink
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch_dedupe
> +
> +rm -f $seqres.full
> +
> +nr_ops=$((TIME_FACTOR * 10000))
> +
> +# Format filesystem
> +_scratch_mkfs > $seqres.full
> +_scratch_mount
> +
> +# Test once with mmap writes
> +$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops
> +
> +# Test again with pwrites for the lulz
> +$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops -w
> +
> +echo Silence is golden.
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/949.out b/tests/generic/949.out
> new file mode 100644
> index 00000000..2998b46c
> --- /dev/null
> +++ b/tests/generic/949.out
> @@ -0,0 +1,2 @@
> +QA output created by 949
> +Silence is golden.
> diff --git a/tests/generic/group b/tests/generic/group
> index d5cfdd51..778aa8c4 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -630,3 +630,4 @@
>  625 auto quick verity
>  947 auto quick rw clone
>  948 auto quick rw copy_range
> +949 auto quick rw dedupe clone


-- 
chandan

Re: [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails

[Chandan Babu R]

On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> This is a regression test for a bug in the XFS implementation of
> FSSETXATTR.  When we try to change a file's project id, the quota
> reservation code will update the incore quota reservations for delayed
> allocation blocks.  Unfortunately, it does this before we finish
> validating all the FSSETXATTR parameters, which means that if we decide
> to bail out, we also fail to undo the incore changes.
>

Looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  .gitignore          |    1 +
>  src/Makefile        |    2 +
>  src/chprojid_fail.c |   92 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/765       |   71 +++++++++++++++++++++++++++++++++++++++
>  tests/xfs/765.out   |    4 ++
>  tests/xfs/group     |    1 +
>  6 files changed, 170 insertions(+), 1 deletion(-)
>  create mode 100644 src/chprojid_fail.c
>  create mode 100755 tests/xfs/765
>  create mode 100644 tests/xfs/765.out
>
>
> diff --git a/.gitignore b/.gitignore
> index 03c03be5..3af8e207 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -58,6 +58,7 @@
>  /src/bulkstat_null_ocount
>  /src/bulkstat_unlink_test
>  /src/bulkstat_unlink_test_modified
> +/src/chprojid_fail
>  /src/cloner
>  /src/dbtest
>  /src/devzero
> diff --git a/src/Makefile b/src/Makefile
> index 38ee6718..3d729a34 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -29,7 +29,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
>  	attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
>  	dio-invalidate-cache stat_test t_encrypted_d_revalidate \
>  	attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
> -	fscrypt-crypt-util bulkstat_null_ocount splice-test
> +	fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail
>  
>  SUBDIRS = log-writes perf
>  
> diff --git a/src/chprojid_fail.c b/src/chprojid_fail.c
> new file mode 100644
> index 00000000..8c5b5fee
> --- /dev/null
> +++ b/src/chprojid_fail.c
> @@ -0,0 +1,92 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2021 Oracle.  All Rights Reserved.
> + * Author: Darrick J. Wong <djwong@kernel.org>
> + *
> + * Regression test for failing to undo delalloc quota reservations when
> + * changing project id and we fail some other FSSETXATTR validation.
> + */
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <unistd.h>
> +#include <sys/ioctl.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <errno.h>
> +#include <linux/fs.h>
> +
> +static char zerobuf[65536];
> +
> +int
> +main(
> +	int		argc,
> +	char		*argv[])
> +{
> +	struct fsxattr	fa;
> +	ssize_t		sz;
> +	int		fd, ret;
> +
> +	if (argc < 2) {
> +		printf("Usage: %s filename\n", argv[0]);
> +		return 1;
> +	}
> +
> +	fd = open(argv[1], O_CREAT | O_TRUNC | O_RDWR, 0600);
> +	if (fd < 0) {
> +		perror(argv[1]);
> +		return 2;
> +	}
> +
> +	/* Zero the project id and the extent size hint. */
> +	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
> +	if (ret) {
> +		perror("FSGETXATTR check file");
> +		return 2;
> +	}
> +
> +	if (fa.fsx_projid != 0 || fa.fsx_extsize != 0) {
> +		fa.fsx_projid = 0;
> +		fa.fsx_extsize = 0;
> +		ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
> +		if (ret) {
> +			perror("FSSETXATTR zeroing");
> +			return 2;
> +		}
> +	}
> +
> +	/* Dirty a few kb of a file to create delalloc extents. */
> +	sz = write(fd, zerobuf, sizeof(zerobuf));
> +	if (sz != sizeof(zerobuf)) {
> +		perror("delalloc write");
> +		return 2;
> +	}
> +
> +	/*
> +	 * The regression we're trying to test happens when the fsxattr input
> +	 * validation decides to bail out after the chown quota reservation has
> +	 * been made on a file containing delalloc extents.  Extent size hints
> +	 * can't be set on non-empty files and we can't check the value until
> +	 * we've reserved resources and taken the file's ILOCK, so this is a
> +	 * perfect vector for triggering this condition.  In this way we set up
> +	 * a FSSETXATTR call that will fail.
> +	 */
> +	ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
> +	if (ret) {
> +		perror("FSGETXATTR");
> +		return 2;
> +	}
> +
> +	fa.fsx_projid = 23652;
> +	fa.fsx_extsize = 2;
> +	fa.fsx_xflags |= FS_XFLAG_EXTSIZE;
> +
> +	ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
> +	if (ret) {
> +		printf("FSSETXATTRR should fail: %s\n", strerror(errno));
> +		return 0;
> +	}
> +
> +	/* Uhoh, that FSSETXATTR call should have failed! */
> +	return 3;
> +}
> diff --git a/tests/xfs/765 b/tests/xfs/765
> new file mode 100755
> index 00000000..68b89ce3
> --- /dev/null
> +++ b/tests/xfs/765
> @@ -0,0 +1,71 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 765
> +#
> +# Regression test for failing to undo delalloc quota reservations when changing
> +# project id but we fail some other part of FSSETXATTR validation.  If we fail
> +# the test, we trip debugging assertions in dmesg.  This is a regression test
> +# for commit 1aecf3734a95 ("xfs: fix chown leaking delalloc quota blocks when
> +# fssetxattr fails").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/quota
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_command "$FILEFRAG_PROG" filefrag
> +_require_test_program "chprojid_fail"
> +_require_quota
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +echo "Format filesystem" | tee -a $seqres.full
> +_scratch_mkfs > $seqres.full
> +_qmount_option 'prjquota'
> +_qmount
> +_require_prjquota $SCRATCH_DEV
> +
> +# Make sure that a regular buffered write produces delalloc reservations.
> +$XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null
> +$FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \
> +	_notrun "test requires delayed allocation writes"
> +rm -f $SCRATCH_MNT/testy
> +
> +echo "Run test program"
> +$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
> +$here/src/chprojid_fail $SCRATCH_MNT/blah
> +
> +# The regression we're testing for is an accounting bug involving delalloc
> +# reservations.  FSSETXATTR does not itself cause dirty data writeback, so we
> +# assume that if the file still has delalloc extents, then it must have had
> +# them when chprojid_fail was running, and therefore the test was set up
> +# correctly.  There's a slight chance that background writeback can sneak in
> +# and flush the file, but this should be a small enough gap.
> +$FILEFRAG_PROG -v $SCRATCH_MNT/blah 2>&1 | grep -q delalloc || \
> +	echo "file didn't get delalloc extents, test invalid?"
> +
> +# Make a note of current quota status for diagnostic purposes
> +$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/765.out b/tests/xfs/765.out
> new file mode 100644
> index 00000000..d5f8fc11
> --- /dev/null
> +++ b/tests/xfs/765.out
> @@ -0,0 +1,4 @@
> +QA output created by 765
> +Format filesystem
> +Run test program
> +FSSETXATTRR should fail: Invalid argument
> diff --git a/tests/xfs/group b/tests/xfs/group
> index d7aafc94..84468d10 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -505,4 +505,5 @@
>  760 auto quick rw collapse punch insert zero prealloc
>  761 auto quick realtime
>  763 auto quick rw realtime
> +765 auto quick quota
>  915 auto quick quota


-- 
chandan

Re: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes

[Eryu Guan]

On Mon, Mar 08, 2021 at 08:40:25PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> In "mkfs: set required parts of the realtime geometry before computing
> log geometry" we made sure that mkfs set up enough of the fs geometry to
> compute the minimum xfs log size correctly when formatting the
> filesystem.  This is the regression test for that issue.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/761     |   45 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/761.out |    1 +
>  tests/xfs/group   |    1 +
>  3 files changed, 47 insertions(+)
>  create mode 100755 tests/xfs/761
>  create mode 100644 tests/xfs/761.out
> 
> 
> diff --git a/tests/xfs/761 b/tests/xfs/761
> new file mode 100755
> index 00000000..b9770d90
> --- /dev/null
> +++ b/tests/xfs/761
> @@ -0,0 +1,45 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 761
> +#
> +# Make sure mkfs sets up enough of the rt geometry that we can compute the
> +# correct min log size for formatting the fs.
> +#
> +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
> +# required parts of the realtime geometry before computing log geometry").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_realtime
> +
> +rm -f $seqres.full
> +
> +# Format a tiny filesystem to force minimum log size, then see if it mounts
> +_scratch_mkfs -r size=1m -d size=100m > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/761.out b/tests/xfs/761.out
> new file mode 100644
> index 00000000..8c9d9e90
> --- /dev/null
> +++ b/tests/xfs/761.out
> @@ -0,0 +1 @@
> +QA output created by 761

Need "Silence is golden" in .out file if you're going to respin it,
otherwise I can fix it on commit.

Thanks,
Eryu

> diff --git a/tests/xfs/group b/tests/xfs/group
> index 318468b5..87badd56 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -503,4 +503,5 @@
>  758 auto quick rw attr realtime
>  759 auto quick rw realtime
>  760 auto quick rw collapse punch insert zero prealloc
> +761 auto quick realtime
>  763 auto quick rw realtime

Re: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes

[Eryu Guan]

On Sun, Mar 14, 2021 at 10:51:41PM +0800, Eryu Guan wrote:
> On Mon, Mar 08, 2021 at 08:40:25PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > In "mkfs: set required parts of the realtime geometry before computing
> > log geometry" we made sure that mkfs set up enough of the fs geometry to
> > compute the minimum xfs log size correctly when formatting the
> > filesystem.  This is the regression test for that issue.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/761     |   45 +++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/761.out |    1 +
> >  tests/xfs/group   |    1 +
> >  3 files changed, 47 insertions(+)
> >  create mode 100755 tests/xfs/761
> >  create mode 100644 tests/xfs/761.out
> > 
> > 
> > diff --git a/tests/xfs/761 b/tests/xfs/761
> > new file mode 100755
> > index 00000000..b9770d90
> > --- /dev/null
> > +++ b/tests/xfs/761
> > @@ -0,0 +1,45 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 761
> > +#
> > +# Make sure mkfs sets up enough of the rt geometry that we can compute the
> > +# correct min log size for formatting the fs.
> > +#
> > +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
> > +# required parts of the realtime geometry before computing log geometry").
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_require_scratch
> > +_require_realtime
> > +
> > +rm -f $seqres.full
> > +
> > +# Format a tiny filesystem to force minimum log size, then see if it mounts
> > +_scratch_mkfs -r size=1m -d size=100m > $seqres.full
> > +_scratch_mount >> $seqres.full
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/xfs/761.out b/tests/xfs/761.out
> > new file mode 100644
> > index 00000000..8c9d9e90
> > --- /dev/null
> > +++ b/tests/xfs/761.out
> > @@ -0,0 +1 @@
> > +QA output created by 761
> 
> Need "Silence is golden" in .out file if you're going to respin it,
> otherwise I can fix it on commit.

Never mind, I've applied it with .out file fixed, and added 'mount'
group as Chandan suggested.

Thanks,
Eryu

Re: [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files

[Eryu Guan]

On Mon, Mar 08, 2021 at 08:40:53PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> wenli xie reported a buffer cache deadlock when an overlayfs is mounted
> atop xfs and overlayfs tries to replace a single-nlink file with a
> whiteout file.  This test reproduces that deadlock.
> 
> Reported-by: wenli xie <wlxie7296@gmail.com>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/generic/1300     |  109 ++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/1300.out |    2 +
>  tests/generic/group    |    1 
>  3 files changed, 112 insertions(+)
>  create mode 100755 tests/generic/1300
>  create mode 100644 tests/generic/1300.out
> 
> 
> diff --git a/tests/generic/1300 b/tests/generic/1300
> new file mode 100755
> index 00000000..10df44e3
> --- /dev/null
> +++ b/tests/generic/1300
> @@ -0,0 +1,109 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 1300
> +#
> +# Reproducer for a deadlock in xfs_rename reported by Wenli Xie.
> +#
> +# When overlayfs is running on top of xfs and the user unlinks a file in the
> +# overlay, overlayfs will create a whiteout inode and ask us to "rename" the
> +# whiteout file atop the one being unlinked.  If the file being unlinked loses
> +# its one nlink, we then have to put the inode on the unlinked list.
> +#
> +# This requires us to grab the AGI buffer of the whiteout inode to take it
> +# off the unlinked list (which is where whiteouts are created) and to grab
> +# the AGI buffer of the file being deleted.  If the whiteout was created in
> +# a higher numbered AG than the file being deleted, we'll lock the AGIs in
> +# the wrong order and deadlock.
> +#
> +# Note that this test doesn't do anything xfs-specific so it's a generic test.
> +# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock
> +# in xfs_rename").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	stop_workers
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch
> +test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs."
> +
> +modprobe -q overlay
> +grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs."

We have _require_ext2() and _require_tmpfs(), I think it's time to
refactor them into a new _require_filesystem() helper?

> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >> $seqres.full
> +_scratch_mount

Also, doing overlay mount requires the underlying to export d_type,
perhaps we should do '_supports_filetype $SCRATCH_MNT' here, otherwise
filesystems like nfs would fail the test.

> +
> +mkdir $SCRATCH_MNT/lowerdir
> +mkdir $SCRATCH_MNT/lowerdir1
> +mkdir $SCRATCH_MNT/lowerdir/etc
> +mkdir $SCRATCH_MNT/workers
> +echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf
> +touch $SCRATCH_MNT/running
> +
> +stop_workers() {
> +	test -e $SCRATCH_MNT/running || return
> +	rm -f $SCRATCH_MNT/running
> +
> +	while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do
> +		wait
> +	done
> +}
> +
> +worker() {
> +	local tag="$1"
> +	local mergedir="$SCRATCH_MNT/merged$tag"
> +	local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1"
> +	local u="upperdir=$SCRATCH_MNT/upperdir$tag"
> +	local w="workdir=$SCRATCH_MNT/workdir$tag"
> +	local i="index=off,nfs_export=off"
> +
> +	touch $SCRATCH_MNT/workers/$tag
> +	while test -e $SCRATCH_MNT/running; do
> +		rm -rf $SCRATCH_MNT/merged$tag
> +		rm -rf $SCRATCH_MNT/upperdir$tag
> +		rm -rf $SCRATCH_MNT/workdir$tag
> +		mkdir $SCRATCH_MNT/merged$tag
> +		mkdir $SCRATCH_MNT/workdir$tag
> +		mkdir $SCRATCH_MNT/upperdir$tag
> +
> +		mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir
> +		mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> +		touch $mergedir/etc/access.conf
> +		mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> +		touch $mergedir/etc/access.conf
> +		umount $mergedir
> +	done
> +	rm -f $SCRATCH_MNT/workers/$tag
> +}
> +
> +for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do
> +	worker $i &
> +done
> +
> +sleep $((30 * TIME_FACTOR))
> +stop_workers
> +
> +echo Silence is golden.
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/1300.out b/tests/generic/1300.out
> new file mode 100644
> index 00000000..5805d30d
> --- /dev/null
> +++ b/tests/generic/1300.out
> @@ -0,0 +1,2 @@
> +QA output created by 1300
> +Silence is golden.
> diff --git a/tests/generic/group b/tests/generic/group
> index 778aa8c4..2233a59d 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -631,3 +631,4 @@
>  947 auto quick rw clone
>  948 auto quick rw copy_range
>  949 auto quick rw dedupe clone
> +1300 auto rw overlay

Also add 'rename' group?

Thanks,
Eryu

Re: [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12

[Eryu Guan]

On Mon, Mar 08, 2021 at 08:40:03PM -0800, Darrick J. Wong wrote:
> Hi all,
> 
> Here are new tests for problems that were fixed in upstream Linux
> between 5.9 and 5.12.

I've applied all patches except 3/10 and 9/10, thanks for the tests and
fixes!

Eryu

> 
> If you're going to start using this mess, you probably ought to just
> pull from my git trees, which are linked below.
> 
> This is an extraordinary way to destroy everything.  Enjoy!
> Comments and questions are, as always, welcome.
> 
> --D
> 
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=kernel-regressions
> ---
>  .gitignore             |    1 
>  common/filter          |   24 +++
>  src/Makefile           |    4 -
>  src/chprojid_fail.c    |   92 ++++++++++++
>  src/deduperace.c       |  370 ++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/1300     |  109 ++++++++++++++
>  tests/generic/1300.out |    2 
>  tests/generic/947      |  118 +++++++++++++++
>  tests/generic/947.out  |   15 ++
>  tests/generic/948      |   92 ++++++++++++
>  tests/generic/948.out  |    9 +
>  tests/generic/949      |   51 +++++++
>  tests/generic/949.out  |    2 
>  tests/generic/group    |    4 +
>  tests/xfs/050          |   30 +---
>  tests/xfs/122          |    1 
>  tests/xfs/122.out      |    1 
>  tests/xfs/299          |   30 +---
>  tests/xfs/758          |   59 ++++++++
>  tests/xfs/758.out      |    2 
>  tests/xfs/759          |  100 +++++++++++++
>  tests/xfs/759.out      |    2 
>  tests/xfs/760          |   68 +++++++++
>  tests/xfs/760.out      |    9 +
>  tests/xfs/761          |   45 ++++++
>  tests/xfs/761.out      |    1 
>  tests/xfs/765          |   71 +++++++++
>  tests/xfs/765.out      |    4 +
>  tests/xfs/915          |  162 +++++++++++++++++++++
>  tests/xfs/915.out      |  151 ++++++++++++++++++++
>  tests/xfs/group        |    6 +
>  31 files changed, 1584 insertions(+), 51 deletions(-)
>  create mode 100644 src/chprojid_fail.c
>  create mode 100644 src/deduperace.c
>  create mode 100755 tests/generic/1300
>  create mode 100644 tests/generic/1300.out
>  create mode 100755 tests/generic/947
>  create mode 100644 tests/generic/947.out
>  create mode 100755 tests/generic/948
>  create mode 100644 tests/generic/948.out
>  create mode 100755 tests/generic/949
>  create mode 100644 tests/generic/949.out
>  create mode 100755 tests/xfs/758
>  create mode 100644 tests/xfs/758.out
>  create mode 100755 tests/xfs/759
>  create mode 100644 tests/xfs/759.out
>  create mode 100755 tests/xfs/760
>  create mode 100644 tests/xfs/760.out
>  create mode 100755 tests/xfs/761
>  create mode 100644 tests/xfs/761.out
>  create mode 100755 tests/xfs/765
>  create mode 100644 tests/xfs/765.out
>  create mode 100755 tests/xfs/915
>  create mode 100644 tests/xfs/915.out

Re: [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12

[Darrick J. Wong]

On Mon, Mar 15, 2021 at 02:07:44AM +0800, Eryu Guan wrote:
> On Mon, Mar 08, 2021 at 08:40:03PM -0800, Darrick J. Wong wrote:
> > Hi all,
> > 
> > Here are new tests for problems that were fixed in upstream Linux
> > between 5.9 and 5.12.
> 
> I've applied all patches except 3/10 and 9/10, thanks for the tests and
> fixes!

Thank /you/ for putting them in!  I'll revise those two and have a new
submission in a few days.

--D

> Eryu
> 
> > 
> > If you're going to start using this mess, you probably ought to just
> > pull from my git trees, which are linked below.
> > 
> > This is an extraordinary way to destroy everything.  Enjoy!
> > Comments and questions are, as always, welcome.
> > 
> > --D
> > 
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=kernel-regressions
> > ---
> >  .gitignore             |    1 
> >  common/filter          |   24 +++
> >  src/Makefile           |    4 -
> >  src/chprojid_fail.c    |   92 ++++++++++++
> >  src/deduperace.c       |  370 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/1300     |  109 ++++++++++++++
> >  tests/generic/1300.out |    2 
> >  tests/generic/947      |  118 +++++++++++++++
> >  tests/generic/947.out  |   15 ++
> >  tests/generic/948      |   92 ++++++++++++
> >  tests/generic/948.out  |    9 +
> >  tests/generic/949      |   51 +++++++
> >  tests/generic/949.out  |    2 
> >  tests/generic/group    |    4 +
> >  tests/xfs/050          |   30 +---
> >  tests/xfs/122          |    1 
> >  tests/xfs/122.out      |    1 
> >  tests/xfs/299          |   30 +---
> >  tests/xfs/758          |   59 ++++++++
> >  tests/xfs/758.out      |    2 
> >  tests/xfs/759          |  100 +++++++++++++
> >  tests/xfs/759.out      |    2 
> >  tests/xfs/760          |   68 +++++++++
> >  tests/xfs/760.out      |    9 +
> >  tests/xfs/761          |   45 ++++++
> >  tests/xfs/761.out      |    1 
> >  tests/xfs/765          |   71 +++++++++
> >  tests/xfs/765.out      |    4 +
> >  tests/xfs/915          |  162 +++++++++++++++++++++
> >  tests/xfs/915.out      |  151 ++++++++++++++++++++
> >  tests/xfs/group        |    6 +
> >  31 files changed, 1584 insertions(+), 51 deletions(-)
> >  create mode 100644 src/chprojid_fail.c
> >  create mode 100644 src/deduperace.c
> >  create mode 100755 tests/generic/1300
> >  create mode 100644 tests/generic/1300.out
> >  create mode 100755 tests/generic/947
> >  create mode 100644 tests/generic/947.out
> >  create mode 100755 tests/generic/948
> >  create mode 100644 tests/generic/948.out
> >  create mode 100755 tests/generic/949
> >  create mode 100644 tests/generic/949.out
> >  create mode 100755 tests/xfs/758
> >  create mode 100644 tests/xfs/758.out
> >  create mode 100755 tests/xfs/759
> >  create mode 100644 tests/xfs/759.out
> >  create mode 100755 tests/xfs/760
> >  create mode 100644 tests/xfs/760.out
> >  create mode 100755 tests/xfs/761
> >  create mode 100644 tests/xfs/761.out
> >  create mode 100755 tests/xfs/765
> >  create mode 100644 tests/xfs/765.out
> >  create mode 100755 tests/xfs/915
> >  create mode 100644 tests/xfs/915.out

Re: [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files

[Darrick J. Wong]

On Mon, Mar 15, 2021 at 02:06:35AM +0800, Eryu Guan wrote:
> On Mon, Mar 08, 2021 at 08:40:53PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > wenli xie reported a buffer cache deadlock when an overlayfs is mounted
> > atop xfs and overlayfs tries to replace a single-nlink file with a
> > whiteout file.  This test reproduces that deadlock.
> > 
> > Reported-by: wenli xie <wlxie7296@gmail.com>
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/generic/1300     |  109 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/1300.out |    2 +
> >  tests/generic/group    |    1 
> >  3 files changed, 112 insertions(+)
> >  create mode 100755 tests/generic/1300
> >  create mode 100644 tests/generic/1300.out
> > 
> > 
> > diff --git a/tests/generic/1300 b/tests/generic/1300
> > new file mode 100755
> > index 00000000..10df44e3
> > --- /dev/null
> > +++ b/tests/generic/1300
> > @@ -0,0 +1,109 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 1300
> > +#
> > +# Reproducer for a deadlock in xfs_rename reported by Wenli Xie.
> > +#
> > +# When overlayfs is running on top of xfs and the user unlinks a file in the
> > +# overlay, overlayfs will create a whiteout inode and ask us to "rename" the
> > +# whiteout file atop the one being unlinked.  If the file being unlinked loses
> > +# its one nlink, we then have to put the inode on the unlinked list.
> > +#
> > +# This requires us to grab the AGI buffer of the whiteout inode to take it
> > +# off the unlinked list (which is where whiteouts are created) and to grab
> > +# the AGI buffer of the file being deleted.  If the whiteout was created in
> > +# a higher numbered AG than the file being deleted, we'll lock the AGIs in
> > +# the wrong order and deadlock.
> > +#
> > +# Note that this test doesn't do anything xfs-specific so it's a generic test.
> > +# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock
> > +# in xfs_rename").
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	stop_workers
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_scratch
> > +test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs."
> > +
> > +modprobe -q overlay
> > +grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs."
> 
> We have _require_ext2() and _require_tmpfs(), I think it's time to
> refactor them into a new _require_filesystem() helper?

Yeah, I'll refactor them into a new _require_extra_fs helper.

> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs >> $seqres.full
> > +_scratch_mount
> 
> Also, doing overlay mount requires the underlying to export d_type,
> perhaps we should do '_supports_filetype $SCRATCH_MNT' here, otherwise
> filesystems like nfs would fail the test.

Oh!  Thank you for pointing that out.
> 
> > +
> > +mkdir $SCRATCH_MNT/lowerdir
> > +mkdir $SCRATCH_MNT/lowerdir1
> > +mkdir $SCRATCH_MNT/lowerdir/etc
> > +mkdir $SCRATCH_MNT/workers
> > +echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf
> > +touch $SCRATCH_MNT/running
> > +
> > +stop_workers() {
> > +	test -e $SCRATCH_MNT/running || return
> > +	rm -f $SCRATCH_MNT/running
> > +
> > +	while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do
> > +		wait
> > +	done
> > +}
> > +
> > +worker() {
> > +	local tag="$1"
> > +	local mergedir="$SCRATCH_MNT/merged$tag"
> > +	local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1"
> > +	local u="upperdir=$SCRATCH_MNT/upperdir$tag"
> > +	local w="workdir=$SCRATCH_MNT/workdir$tag"
> > +	local i="index=off,nfs_export=off"
> > +
> > +	touch $SCRATCH_MNT/workers/$tag
> > +	while test -e $SCRATCH_MNT/running; do
> > +		rm -rf $SCRATCH_MNT/merged$tag
> > +		rm -rf $SCRATCH_MNT/upperdir$tag
> > +		rm -rf $SCRATCH_MNT/workdir$tag
> > +		mkdir $SCRATCH_MNT/merged$tag
> > +		mkdir $SCRATCH_MNT/workdir$tag
> > +		mkdir $SCRATCH_MNT/upperdir$tag
> > +
> > +		mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir
> > +		mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> > +		touch $mergedir/etc/access.conf
> > +		mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> > +		touch $mergedir/etc/access.conf
> > +		umount $mergedir
> > +	done
> > +	rm -f $SCRATCH_MNT/workers/$tag
> > +}
> > +
> > +for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do
> > +	worker $i &
> > +done
> > +
> > +sleep $((30 * TIME_FACTOR))
> > +stop_workers
> > +
> > +echo Silence is golden.
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/1300.out b/tests/generic/1300.out
> > new file mode 100644
> > index 00000000..5805d30d
> > --- /dev/null
> > +++ b/tests/generic/1300.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 1300
> > +Silence is golden.
> > diff --git a/tests/generic/group b/tests/generic/group
> > index 778aa8c4..2233a59d 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -631,3 +631,4 @@
> >  947 auto quick rw clone
> >  948 auto quick rw copy_range
> >  949 auto quick rw dedupe clone
> > +1300 auto rw overlay
> 
> Also add 'rename' group?

Done.

--D

> Thanks,
> Eryu

Re: [PATCH 03/10] xfs: test rtalloc alignment and math errors

[Darrick J. Wong]

On Thu, Mar 11, 2021 at 01:28:32PM +0530, Chandan Babu R wrote:
> On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > Add a couple of regression tests for "xfs: make sure the rt allocator
> > doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
> > finsert operations are aligned to rt extent size".
> >
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/759     |  100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/759.out |    2 +
> >  tests/xfs/760     |   68 ++++++++++++++++++++++++++++++++++++
> >  tests/xfs/760.out |    9 +++++
> >  tests/xfs/group   |    2 +
> >  5 files changed, 181 insertions(+)
> >  create mode 100755 tests/xfs/759
> >  create mode 100644 tests/xfs/759.out
> >  create mode 100755 tests/xfs/760
> >  create mode 100644 tests/xfs/760.out
> >
> >
> > diff --git a/tests/xfs/759 b/tests/xfs/759
> > new file mode 100755
> > index 00000000..8558fe30
> > --- /dev/null
> > +++ b/tests/xfs/759
> > @@ -0,0 +1,100 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 759
> > +#
> > +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
> > +# allocator doesn't run off the end") which fixes an overflow error in the
> > +# _near realtime allocator.  If the rt bitmap ends exactly at the end of a
> > +# block and the number of rt extents is large enough to allow an allocation
> > +# request larger than the maximum extent size, it's possible that during a
> > +# large allocation request, the allocator will fail to constrain maxlen on the
> > +# second run through the loop, and the rt bitmap range check will run right off
> > +# the end of the rtbitmap file.  When this happens, xfs triggers a verifier
> > +# error and returns EFSCORRUPTED.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_require_scratch
> > +_require_realtime
> > +_require_test_program "punch-alternating"
> > +
> > +rm -f $seqres.full
> > +
> > +# Format filesystem to get the block size
> > +_scratch_mkfs > $seqres.full
> > +_scratch_mount >> $seqres.full
> > +
> > +blksz=$(_get_block_size $SCRATCH_MNT)
> > +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
> > +rextblks=$((rextsize / blksz))
> > +
> > +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
> > +
> > +_scratch_unmount
> > +
> > +# Format filesystem with a realtime volume whose size fits the following:
> > +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
> 
> Shouldn't the multiplier be RT extent size rather than FS block size?

No, because reproducing the bug requires a large enough allocation
request that we can't fit it all in a single data fork extent mapping
and have to call back into the allocator to get more space.  bmbt
mappings are always in units of fsblocks, not rt extents.

> > +# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes.
> 
> i.e The bits in one rt bitmap block map (NBBY * blksz * rextsize) bytes of an
> rt device. Hence to have the bitmap end at a fs block boundary the
> corresponding rt device size should be a multiple of this product. Is my
> understanding correct?

Right.

(I swear I responded to this but who knows these days...)

--D

> 
> > +
> > +rtsize1=$((2097151 * blksz))
> > +rtsize2=$((8 * blksz * rextsize))
> > +rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 ))
> > +
> > +echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full
> > +
> > +test $rtsize -gt $rtsize1 || \
> > +	_notrun "scratch rt device too small, need $rtsize1 bytes"
> > +test $rtsize -gt $rtsize2 || \
> > +	_notrun "scratch rt device too small, need $rtsize2 bytes"
> > +
> > +rtsize=$((rtsize - (rtsize % rtsize2)))
> > +
> 
> --
> chandan

Re: [PATCH 03/10] xfs: test rtalloc alignment and math errors

[Chandan Babu R]

On 23 Mar 2021 at 09:45, Darrick J. Wong wrote:
> On Thu, Mar 11, 2021 at 01:28:32PM +0530, Chandan Babu R wrote:
>> On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
>> > From: Darrick J. Wong <djwong@kernel.org>
>> >
>> > Add a couple of regression tests for "xfs: make sure the rt allocator
>> > doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
>> > finsert operations are aligned to rt extent size".
>> >
>> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>> > ---
>> >  tests/xfs/759     |  100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >  tests/xfs/759.out |    2 +
>> >  tests/xfs/760     |   68 ++++++++++++++++++++++++++++++++++++
>> >  tests/xfs/760.out |    9 +++++
>> >  tests/xfs/group   |    2 +
>> >  5 files changed, 181 insertions(+)
>> >  create mode 100755 tests/xfs/759
>> >  create mode 100644 tests/xfs/759.out
>> >  create mode 100755 tests/xfs/760
>> >  create mode 100644 tests/xfs/760.out
>> >
>> >
>> > diff --git a/tests/xfs/759 b/tests/xfs/759
>> > new file mode 100755
>> > index 00000000..8558fe30
>> > --- /dev/null
>> > +++ b/tests/xfs/759
>> > @@ -0,0 +1,100 @@
>> > +#! /bin/bash
>> > +# SPDX-License-Identifier: GPL-2.0-or-later
>> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
>> > +#
>> > +# FS QA Test No. 759
>> > +#
>> > +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
>> > +# allocator doesn't run off the end") which fixes an overflow error in the
>> > +# _near realtime allocator.  If the rt bitmap ends exactly at the end of a
>> > +# block and the number of rt extents is large enough to allow an allocation
>> > +# request larger than the maximum extent size, it's possible that during a
>> > +# large allocation request, the allocator will fail to constrain maxlen on the
>> > +# second run through the loop, and the rt bitmap range check will run right off
>> > +# the end of the rtbitmap file.  When this happens, xfs triggers a verifier
>> > +# error and returns EFSCORRUPTED.
>> > +
>> > +seq=`basename $0`
>> > +seqres=$RESULT_DIR/$seq
>> > +echo "QA output created by $seq"
>> > +
>> > +here=`pwd`
>> > +tmp=/tmp/$$
>> > +status=1    # failure is the default!
>> > +trap "_cleanup; exit \$status" 0 1 2 3 15
>> > +
>> > +_cleanup()
>> > +{
>> > +	cd /
>> > +	rm -f $tmp.*
>> > +}
>> > +
>> > +# get standard environment, filters and checks
>> > +. ./common/rc
>> > +. ./common/filter
>> > +
>> > +# real QA test starts here
>> > +_supported_fs xfs
>> > +_require_scratch
>> > +_require_realtime
>> > +_require_test_program "punch-alternating"
>> > +
>> > +rm -f $seqres.full
>> > +
>> > +# Format filesystem to get the block size
>> > +_scratch_mkfs > $seqres.full
>> > +_scratch_mount >> $seqres.full
>> > +
>> > +blksz=$(_get_block_size $SCRATCH_MNT)
>> > +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
>> > +rextblks=$((rextsize / blksz))
>> > +
>> > +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
>> > +
>> > +_scratch_unmount
>> > +
>> > +# Format filesystem with a realtime volume whose size fits the following:
>> > +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
>> 
>> Shouldn't the multiplier be RT extent size rather than FS block size?
>
> No, because reproducing the bug requires a large enough allocation
> request that we can't fit it all in a single data fork extent mapping
> and have to call back into the allocator to get more space.  bmbt
> mappings are always in units of fsblocks, not rt extents.
>

Ah, I had not realized that.

The changes look good to me,

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

-- 
chandan