* [PATCH 01/10] xfs: test regression in xfs_bmap_validate_extent
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-09 4:40 ` [PATCH 02/10] generic: test reflink and copy_file_range behavior with O_SYNC and FS_XFLAG_SYNC files Darrick J. Wong
` (9 subsequent siblings)
10 siblings, 0 replies; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: Chandan Babu R, linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
This is a regression test to make sure that we can have realtime files
with xattr blocks and not trip the verifiers.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
---
tests/xfs/758 | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/758.out | 2 ++
tests/xfs/group | 1 +
3 files changed, 62 insertions(+)
create mode 100755 tests/xfs/758
create mode 100644 tests/xfs/758.out
diff --git a/tests/xfs/758 b/tests/xfs/758
new file mode 100755
index 00000000..a247701e
--- /dev/null
+++ b/tests/xfs/758
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 758
+#
+# This is a regression test for commit d0c20d38af13 "xfs: fix
+# xfs_bmap_validate_extent_raw when checking attr fork of rt files", which
+# fixes the bmap record validator so that it will not check the attr fork
+# extent mappings of a realtime file against the size of the realtime volume.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+
+rm -f $seqres.full
+
+# Format filesystem with very tiny realtime volume
+_scratch_mkfs -r size=256k > $seqres.full
+_scratch_mount >> $seqres.full
+
+# Create a realtime file
+$XFS_IO_PROG -f -R -c 'pwrite 0 64k' -c stat $SCRATCH_MNT/v >> $seqres.full
+
+# Add enough xattr data to force creation of xattr blocks at a higher address
+# on the data device than the size of the realtime volume
+for i in `seq 0 16`; do
+ $ATTR_PROG -s user.test$i $SCRATCH_MNT/v < $SCRATCH_MNT/v >> $seqres.full
+done
+
+# Force flushing extent maps to disk to trip the verifier
+_scratch_cycle_mount
+
+# Now let that unmount
+echo Silence is golden.
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/758.out b/tests/xfs/758.out
new file mode 100644
index 00000000..6d219f8e
--- /dev/null
+++ b/tests/xfs/758.out
@@ -0,0 +1,2 @@
+QA output created by 758
+Silence is golden.
diff --git a/tests/xfs/group b/tests/xfs/group
index 754fc7be..4dd9901f 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -500,4 +500,5 @@
525 auto quick mkfs
526 auto quick mkfs
527 auto quick quota
+758 auto quick rw attr realtime
763 auto quick rw realtime
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 02/10] generic: test reflink and copy_file_range behavior with O_SYNC and FS_XFLAG_SYNC files
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
2021-03-09 4:40 ` [PATCH 01/10] xfs: test regression in xfs_bmap_validate_extent Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-09 4:40 ` [PATCH 03/10] xfs: test rtalloc alignment and math errors Darrick J. Wong
` (8 subsequent siblings)
10 siblings, 0 replies; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: Chandan Babu R, linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
Add two regression tests to make sure that FICLONERANGE and the splice
based copy_file_range actually flush all data and metadata to disk
before the call ends.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
---
tests/generic/947 | 118 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/generic/947.out | 15 ++++++
tests/generic/948 | 92 ++++++++++++++++++++++++++++++++++++++
tests/generic/948.out | 9 ++++
tests/generic/group | 2 +
5 files changed, 236 insertions(+)
create mode 100755 tests/generic/947
create mode 100644 tests/generic/947.out
create mode 100755 tests/generic/948
create mode 100644 tests/generic/948.out
diff --git a/tests/generic/947 b/tests/generic/947
new file mode 100755
index 00000000..d0edb876
--- /dev/null
+++ b/tests/generic/947
@@ -0,0 +1,118 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 947
+#
+# Make sure that reflink forces the log out if we open the file with O_SYNC or
+# set FS_XFLAG_SYNC on the file. We test that it actually forced the log by
+# using dm-error to shut down the fs without flushing the log and then
+# remounting to check file contents. This is a regression test for commit
+# 5ffce3cc22a0 ("xfs: force the log after remapping a synchronous-writes file")
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+ _dmerror_unmount
+ _dmerror_cleanup
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/reflink
+. ./common/dmerror
+
+# real QA test starts here
+_supported_fs generic
+_require_dm_target error
+_require_scratch_reflink
+_require_xfs_io_command "chattr" "s"
+_require_cp_reflink
+
+rm -f $seqres.full
+
+# Format filesystem and set up quota limits
+_scratch_mkfs > $seqres.full
+_require_metadata_journaling $SCRATCH_DEV
+_dmerror_init
+_dmerror_mount
+
+# Test that O_SYNC actually results in file data being written even if the
+# fs immediately dies
+echo "test o_sync write"
+$XFS_IO_PROG -x -f -s -c "pwrite -S 0x58 0 1m -b 1m" $SCRATCH_MNT/0 >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/0 | _filter_scratch
+
+# Set up initial files for reflink test
+$XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m -b 1m' $SCRATCH_MNT/a >> $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x59 0 1m -b 1m' $SCRATCH_MNT/c >> $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/e
+_cp_reflink $SCRATCH_MNT/c $SCRATCH_MNT/d
+touch $SCRATCH_MNT/b
+sync
+
+# Test that reflink forces dirty data/metadata to disk when destination file
+# opened with O_SYNC
+echo "test reflink flag not set o_sync"
+$XFS_IO_PROG -x -s -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# Test that reflink to a shared file forces dirty data/metadata to disk when
+# destination is opened with O_SYNC
+echo "test reflink flag already set o_sync"
+$XFS_IO_PROG -x -s -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/d >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/d | _filter_scratch
+
+# Set up the two files with chattr +S
+rm -f $SCRATCH_MNT/b $SCRATCH_MNT/d
+_cp_reflink $SCRATCH_MNT/c $SCRATCH_MNT/d
+touch $SCRATCH_MNT/b
+chattr +S $SCRATCH_MNT/b $SCRATCH_MNT/d
+sync
+
+# Test that reflink forces dirty data/metadata to disk when destination file
+# has the sync iflag set
+echo "test reflink flag not set iflag"
+$XFS_IO_PROG -x -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# Test that reflink to a shared file forces dirty data/metadata to disk when
+# destination file has the sync iflag set
+echo "test reflink flag already set iflag"
+$XFS_IO_PROG -x -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/d >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/d | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/947.out b/tests/generic/947.out
new file mode 100644
index 00000000..05ba10d1
--- /dev/null
+++ b/tests/generic/947.out
@@ -0,0 +1,15 @@
+QA output created by 947
+test o_sync write
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/0
+test reflink flag not set o_sync
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/b
+test reflink flag already set o_sync
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/d
+test reflink flag not set iflag
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/b
+test reflink flag already set iflag
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/d
diff --git a/tests/generic/948 b/tests/generic/948
new file mode 100755
index 00000000..b79cd279
--- /dev/null
+++ b/tests/generic/948
@@ -0,0 +1,92 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 948
+#
+# Make sure that copy_file_range forces the log out if we open the file with
+# O_SYNC or set FS_XFLAG_SYNC on the file. We test that it actually forced the
+# log by using dm-error to shut down the fs without flushing the log and then
+# remounting to check file contents. This is a regression test for commit
+# 5ffce3cc22a0 ("xfs: force the log after remapping a synchronous-writes file")
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+ _dmerror_unmount
+ _dmerror_cleanup
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/dmerror
+
+# real QA test starts here
+_supported_fs generic
+_require_dm_target error
+_require_xfs_io_command "chattr" "s"
+_require_scratch
+
+rm -f $seqres.full
+
+# Format filesystem and set up quota limits
+_scratch_mkfs > $seqres.full
+_require_metadata_journaling $SCRATCH_DEV
+_dmerror_init
+_dmerror_mount
+
+# Test that O_SYNC actually results in file data being written even if the
+# fs immediately dies
+echo "test o_sync write"
+$XFS_IO_PROG -x -f -s -c "pwrite -S 0x58 0 1m -b 1m" $SCRATCH_MNT/0 >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/0 | _filter_scratch
+
+# Set up initial files for copy test
+$XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m -b 1m' $SCRATCH_MNT/a >> $seqres.full
+touch $SCRATCH_MNT/b
+sync
+
+# Test that unaligned copy file range forces dirty data/metadata to disk when
+# destination file opened with O_SYNC
+echo "test unaligned copy range o_sync"
+$XFS_IO_PROG -x -s -c "copy_range -s 13 -d 13 -l 1048550 $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# Set up dest file with chattr +S
+rm -f $SCRATCH_MNT/b
+touch $SCRATCH_MNT/b
+chattr +S $SCRATCH_MNT/b
+sync
+
+# Test that unaligned copy file range forces dirty data/metadata to disk when
+# destination file has the sync iflag set
+echo "test unaligned copy range iflag"
+$XFS_IO_PROG -x -c "copy_range -s 13 -d 13 -l 1048550 $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full
+_dmerror_load_error_table
+_dmerror_unmount
+_dmerror_load_working_table
+_dmerror_mount
+md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/948.out b/tests/generic/948.out
new file mode 100644
index 00000000..eec6c0dc
--- /dev/null
+++ b/tests/generic/948.out
@@ -0,0 +1,9 @@
+QA output created by 948
+test o_sync write
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/0
+test unaligned copy range o_sync
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a
+2a715d2093b5aca82783a0c5943ac0b8 SCRATCH_MNT/b
+test unaligned copy range iflag
+310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a
+2a715d2093b5aca82783a0c5943ac0b8 SCRATCH_MNT/b
diff --git a/tests/generic/group b/tests/generic/group
index 84db3789..d5cfdd51 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -628,3 +628,5 @@
623 auto quick shutdown
624 auto quick verity
625 auto quick verity
+947 auto quick rw clone
+948 auto quick rw copy_range
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 03/10] xfs: test rtalloc alignment and math errors
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
2021-03-09 4:40 ` [PATCH 01/10] xfs: test regression in xfs_bmap_validate_extent Darrick J. Wong
2021-03-09 4:40 ` [PATCH 02/10] generic: test reflink and copy_file_range behavior with O_SYNC and FS_XFLAG_SYNC files Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-11 7:58 ` Chandan Babu R
2021-03-09 4:40 ` [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes Darrick J. Wong
` (7 subsequent siblings)
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
Add a couple of regression tests for "xfs: make sure the rt allocator
doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
finsert operations are aligned to rt extent size".
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
tests/xfs/759 | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/759.out | 2 +
tests/xfs/760 | 68 ++++++++++++++++++++++++++++++++++++
tests/xfs/760.out | 9 +++++
tests/xfs/group | 2 +
5 files changed, 181 insertions(+)
create mode 100755 tests/xfs/759
create mode 100644 tests/xfs/759.out
create mode 100755 tests/xfs/760
create mode 100644 tests/xfs/760.out
diff --git a/tests/xfs/759 b/tests/xfs/759
new file mode 100755
index 00000000..8558fe30
--- /dev/null
+++ b/tests/xfs/759
@@ -0,0 +1,100 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 759
+#
+# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
+# allocator doesn't run off the end") which fixes an overflow error in the
+# _near realtime allocator. If the rt bitmap ends exactly at the end of a
+# block and the number of rt extents is large enough to allow an allocation
+# request larger than the maximum extent size, it's possible that during a
+# large allocation request, the allocator will fail to constrain maxlen on the
+# second run through the loop, and the rt bitmap range check will run right off
+# the end of the rtbitmap file. When this happens, xfs triggers a verifier
+# error and returns EFSCORRUPTED.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+_require_test_program "punch-alternating"
+
+rm -f $seqres.full
+
+# Format filesystem to get the block size
+_scratch_mkfs > $seqres.full
+_scratch_mount >> $seqres.full
+
+blksz=$(_get_block_size $SCRATCH_MNT)
+rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
+rextblks=$((rextsize / blksz))
+
+echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
+
+_scratch_unmount
+
+# Format filesystem with a realtime volume whose size fits the following:
+# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
+# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes.
+
+rtsize1=$((2097151 * blksz))
+rtsize2=$((8 * blksz * rextsize))
+rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 ))
+
+echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full
+
+test $rtsize -gt $rtsize1 || \
+ _notrun "scratch rt device too small, need $rtsize1 bytes"
+test $rtsize -gt $rtsize2 || \
+ _notrun "scratch rt device too small, need $rtsize2 bytes"
+
+rtsize=$((rtsize - (rtsize % rtsize2)))
+
+echo "rt size will be $rtsize" >> $seqres.full
+
+_scratch_mkfs -r size=$rtsize >> $seqres.full
+_scratch_mount >> $seqres.full
+
+# Make sure the root directory has rtinherit set so our test file will too
+$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+
+# Allocate some stuff at the start, to force the first falloc of the ouch file
+# to happen somewhere in the middle of the rt volume
+$XFS_IO_PROG -f -c 'falloc 0 64m' "$SCRATCH_MNT/b"
+$here/src/punch-alternating -i $((rextblks * 2)) -s $((rextblks)) "$SCRATCH_MNT/b"
+
+avail="$(df -P "$SCRATCH_MNT" | awk 'END {print $4}')"1
+toobig="$((avail * 2))"
+
+# falloc the ouch file in the middle of the rt extent to exercise the near
+# allocator in the last step.
+$XFS_IO_PROG -f -c 'falloc 0 1g' "$SCRATCH_MNT/ouch"
+
+# Try to get the near allocator to overflow on an allocation that matches
+# exactly one of the rtsummary size levels. This should return ENOSPC and
+# not EFSCORRUPTED.
+$XFS_IO_PROG -f -c "falloc 0 ${toobig}k" "$SCRATCH_MNT/ouch"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/759.out b/tests/xfs/759.out
new file mode 100644
index 00000000..df693d50
--- /dev/null
+++ b/tests/xfs/759.out
@@ -0,0 +1,2 @@
+QA output created by 759
+fallocate: No space left on device
diff --git a/tests/xfs/760 b/tests/xfs/760
new file mode 100755
index 00000000..7f2b52d4
--- /dev/null
+++ b/tests/xfs/760
@@ -0,0 +1,68 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 760
+#
+# Make sure we validate realtime extent size alignment for fallocate modes.
+# This is a regression test for fe341eb151ec ("xfs: ensure that fpunch,
+# fcollapse, and finsert operations are aligned to rt extent size")
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+_require_xfs_io_command "fcollapse"
+_require_xfs_io_command "finsert"
+_require_xfs_io_command "funshare"
+_require_xfs_io_command "fzero"
+_require_xfs_io_command "falloc"
+
+rm -f $seqres.full
+
+# Format filesystem with a 256k realtime extent size
+_scratch_mkfs -r extsize=256k > $seqres.full
+_scratch_mount >> $seqres.full
+
+blksz=$(_get_block_size $SCRATCH_MNT)
+rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
+rextblks=$((rextsize / blksz))
+
+echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
+
+# Make sure the root directory has rtinherit set so our test file will too
+$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT
+
+sz=$((rextsize * 100))
+range="$((blksz * 3)) $blksz"
+
+for verb in fpunch finsert fcollapse fzero funshare falloc; do
+ echo "test $verb"
+ $XFS_IO_PROG -f -c "falloc 0 $sz" "$SCRATCH_MNT/b"
+ $XFS_IO_PROG -f -c "$verb $range" "$SCRATCH_MNT/b"
+ rm -f "$SCRATCH_MNT/b"
+ _scratch_cycle_mount
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/760.out b/tests/xfs/760.out
new file mode 100644
index 00000000..3d73c6fa
--- /dev/null
+++ b/tests/xfs/760.out
@@ -0,0 +1,9 @@
+QA output created by 760
+test fpunch
+test finsert
+fallocate: Invalid argument
+test fcollapse
+fallocate: Invalid argument
+test fzero
+test funshare
+test falloc
diff --git a/tests/xfs/group b/tests/xfs/group
index 4dd9901f..318468b5 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -501,4 +501,6 @@
526 auto quick mkfs
527 auto quick quota
758 auto quick rw attr realtime
+759 auto quick rw realtime
+760 auto quick rw collapse punch insert zero prealloc
763 auto quick rw realtime
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 03/10] xfs: test rtalloc alignment and math errors
2021-03-09 4:40 ` [PATCH 03/10] xfs: test rtalloc alignment and math errors Darrick J. Wong
@ 2021-03-11 7:58 ` Chandan Babu R
2021-03-23 4:15 ` Darrick J. Wong
0 siblings, 1 reply; 27+ messages in thread
From: Chandan Babu R @ 2021-03-11 7:58 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Add a couple of regression tests for "xfs: make sure the rt allocator
> doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
> finsert operations are aligned to rt extent size".
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> tests/xfs/759 | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/759.out | 2 +
> tests/xfs/760 | 68 ++++++++++++++++++++++++++++++++++++
> tests/xfs/760.out | 9 +++++
> tests/xfs/group | 2 +
> 5 files changed, 181 insertions(+)
> create mode 100755 tests/xfs/759
> create mode 100644 tests/xfs/759.out
> create mode 100755 tests/xfs/760
> create mode 100644 tests/xfs/760.out
>
>
> diff --git a/tests/xfs/759 b/tests/xfs/759
> new file mode 100755
> index 00000000..8558fe30
> --- /dev/null
> +++ b/tests/xfs/759
> @@ -0,0 +1,100 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 759
> +#
> +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
> +# allocator doesn't run off the end") which fixes an overflow error in the
> +# _near realtime allocator. If the rt bitmap ends exactly at the end of a
> +# block and the number of rt extents is large enough to allow an allocation
> +# request larger than the maximum extent size, it's possible that during a
> +# large allocation request, the allocator will fail to constrain maxlen on the
> +# second run through the loop, and the rt bitmap range check will run right off
> +# the end of the rtbitmap file. When this happens, xfs triggers a verifier
> +# error and returns EFSCORRUPTED.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_realtime
> +_require_test_program "punch-alternating"
> +
> +rm -f $seqres.full
> +
> +# Format filesystem to get the block size
> +_scratch_mkfs > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +blksz=$(_get_block_size $SCRATCH_MNT)
> +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
> +rextblks=$((rextsize / blksz))
> +
> +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
> +
> +_scratch_unmount
> +
> +# Format filesystem with a realtime volume whose size fits the following:
> +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
Shouldn't the multiplier be RT extent size rather than FS block size?
> +# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes.
i.e The bits in one rt bitmap block map (NBBY * blksz * rextsize) bytes of an
rt device. Hence to have the bitmap end at a fs block boundary the
corresponding rt device size should be a multiple of this product. Is my
understanding correct?
> +
> +rtsize1=$((2097151 * blksz))
> +rtsize2=$((8 * blksz * rextsize))
> +rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 ))
> +
> +echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full
> +
> +test $rtsize -gt $rtsize1 || \
> + _notrun "scratch rt device too small, need $rtsize1 bytes"
> +test $rtsize -gt $rtsize2 || \
> + _notrun "scratch rt device too small, need $rtsize2 bytes"
> +
> +rtsize=$((rtsize - (rtsize % rtsize2)))
> +
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 03/10] xfs: test rtalloc alignment and math errors
2021-03-11 7:58 ` Chandan Babu R
@ 2021-03-23 4:15 ` Darrick J. Wong
2021-03-25 7:52 ` Chandan Babu R
0 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-23 4:15 UTC (permalink / raw)
To: Chandan Babu R; +Cc: guaneryu, linux-xfs, fstests, guan
On Thu, Mar 11, 2021 at 01:28:32PM +0530, Chandan Babu R wrote:
> On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > Add a couple of regression tests for "xfs: make sure the rt allocator
> > doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
> > finsert operations are aligned to rt extent size".
> >
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> > tests/xfs/759 | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/759.out | 2 +
> > tests/xfs/760 | 68 ++++++++++++++++++++++++++++++++++++
> > tests/xfs/760.out | 9 +++++
> > tests/xfs/group | 2 +
> > 5 files changed, 181 insertions(+)
> > create mode 100755 tests/xfs/759
> > create mode 100644 tests/xfs/759.out
> > create mode 100755 tests/xfs/760
> > create mode 100644 tests/xfs/760.out
> >
> >
> > diff --git a/tests/xfs/759 b/tests/xfs/759
> > new file mode 100755
> > index 00000000..8558fe30
> > --- /dev/null
> > +++ b/tests/xfs/759
> > @@ -0,0 +1,100 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle. All Rights Reserved.
> > +#
> > +# FS QA Test No. 759
> > +#
> > +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
> > +# allocator doesn't run off the end") which fixes an overflow error in the
> > +# _near realtime allocator. If the rt bitmap ends exactly at the end of a
> > +# block and the number of rt extents is large enough to allow an allocation
> > +# request larger than the maximum extent size, it's possible that during a
> > +# large allocation request, the allocator will fail to constrain maxlen on the
> > +# second run through the loop, and the rt bitmap range check will run right off
> > +# the end of the rtbitmap file. When this happens, xfs triggers a verifier
> > +# error and returns EFSCORRUPTED.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_require_scratch
> > +_require_realtime
> > +_require_test_program "punch-alternating"
> > +
> > +rm -f $seqres.full
> > +
> > +# Format filesystem to get the block size
> > +_scratch_mkfs > $seqres.full
> > +_scratch_mount >> $seqres.full
> > +
> > +blksz=$(_get_block_size $SCRATCH_MNT)
> > +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
> > +rextblks=$((rextsize / blksz))
> > +
> > +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
> > +
> > +_scratch_unmount
> > +
> > +# Format filesystem with a realtime volume whose size fits the following:
> > +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
>
> Shouldn't the multiplier be RT extent size rather than FS block size?
No, because reproducing the bug requires a large enough allocation
request that we can't fit it all in a single data fork extent mapping
and have to call back into the allocator to get more space. bmbt
mappings are always in units of fsblocks, not rt extents.
> > +# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes.
>
> i.e The bits in one rt bitmap block map (NBBY * blksz * rextsize) bytes of an
> rt device. Hence to have the bitmap end at a fs block boundary the
> corresponding rt device size should be a multiple of this product. Is my
> understanding correct?
Right.
(I swear I responded to this but who knows these days...)
--D
>
> > +
> > +rtsize1=$((2097151 * blksz))
> > +rtsize2=$((8 * blksz * rextsize))
> > +rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 ))
> > +
> > +echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full
> > +
> > +test $rtsize -gt $rtsize1 || \
> > + _notrun "scratch rt device too small, need $rtsize1 bytes"
> > +test $rtsize -gt $rtsize2 || \
> > + _notrun "scratch rt device too small, need $rtsize2 bytes"
> > +
> > +rtsize=$((rtsize - (rtsize % rtsize2)))
> > +
>
> --
> chandan
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 03/10] xfs: test rtalloc alignment and math errors
2021-03-23 4:15 ` Darrick J. Wong
@ 2021-03-25 7:52 ` Chandan Babu R
0 siblings, 0 replies; 27+ messages in thread
From: Chandan Babu R @ 2021-03-25 7:52 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 23 Mar 2021 at 09:45, Darrick J. Wong wrote:
> On Thu, Mar 11, 2021 at 01:28:32PM +0530, Chandan Babu R wrote:
>> On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
>> > From: Darrick J. Wong <djwong@kernel.org>
>> >
>> > Add a couple of regression tests for "xfs: make sure the rt allocator
>> > doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and
>> > finsert operations are aligned to rt extent size".
>> >
>> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>> > ---
>> > tests/xfs/759 | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> > tests/xfs/759.out | 2 +
>> > tests/xfs/760 | 68 ++++++++++++++++++++++++++++++++++++
>> > tests/xfs/760.out | 9 +++++
>> > tests/xfs/group | 2 +
>> > 5 files changed, 181 insertions(+)
>> > create mode 100755 tests/xfs/759
>> > create mode 100644 tests/xfs/759.out
>> > create mode 100755 tests/xfs/760
>> > create mode 100644 tests/xfs/760.out
>> >
>> >
>> > diff --git a/tests/xfs/759 b/tests/xfs/759
>> > new file mode 100755
>> > index 00000000..8558fe30
>> > --- /dev/null
>> > +++ b/tests/xfs/759
>> > @@ -0,0 +1,100 @@
>> > +#! /bin/bash
>> > +# SPDX-License-Identifier: GPL-2.0-or-later
>> > +# Copyright (c) 2021 Oracle. All Rights Reserved.
>> > +#
>> > +# FS QA Test No. 759
>> > +#
>> > +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt
>> > +# allocator doesn't run off the end") which fixes an overflow error in the
>> > +# _near realtime allocator. If the rt bitmap ends exactly at the end of a
>> > +# block and the number of rt extents is large enough to allow an allocation
>> > +# request larger than the maximum extent size, it's possible that during a
>> > +# large allocation request, the allocator will fail to constrain maxlen on the
>> > +# second run through the loop, and the rt bitmap range check will run right off
>> > +# the end of the rtbitmap file. When this happens, xfs triggers a verifier
>> > +# error and returns EFSCORRUPTED.
>> > +
>> > +seq=`basename $0`
>> > +seqres=$RESULT_DIR/$seq
>> > +echo "QA output created by $seq"
>> > +
>> > +here=`pwd`
>> > +tmp=/tmp/$$
>> > +status=1 # failure is the default!
>> > +trap "_cleanup; exit \$status" 0 1 2 3 15
>> > +
>> > +_cleanup()
>> > +{
>> > + cd /
>> > + rm -f $tmp.*
>> > +}
>> > +
>> > +# get standard environment, filters and checks
>> > +. ./common/rc
>> > +. ./common/filter
>> > +
>> > +# real QA test starts here
>> > +_supported_fs xfs
>> > +_require_scratch
>> > +_require_realtime
>> > +_require_test_program "punch-alternating"
>> > +
>> > +rm -f $seqres.full
>> > +
>> > +# Format filesystem to get the block size
>> > +_scratch_mkfs > $seqres.full
>> > +_scratch_mount >> $seqres.full
>> > +
>> > +blksz=$(_get_block_size $SCRATCH_MNT)
>> > +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g')
>> > +rextblks=$((rextsize / blksz))
>> > +
>> > +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full
>> > +
>> > +_scratch_unmount
>> > +
>> > +# Format filesystem with a realtime volume whose size fits the following:
>> > +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes.
>>
>> Shouldn't the multiplier be RT extent size rather than FS block size?
>
> No, because reproducing the bug requires a large enough allocation
> request that we can't fit it all in a single data fork extent mapping
> and have to call back into the allocator to get more space. bmbt
> mappings are always in units of fsblocks, not rt extents.
>
Ah, I had not realized that.
The changes look good to me,
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (2 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 03/10] xfs: test rtalloc alignment and math errors Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-11 9:32 ` Chandan Babu R
2021-03-14 14:51 ` Eryu Guan
2021-03-09 4:40 ` [PATCH 05/10] common/filter: refactor quota report filtering Darrick J. Wong
` (6 subsequent siblings)
10 siblings, 2 replies; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
In "mkfs: set required parts of the realtime geometry before computing
log geometry" we made sure that mkfs set up enough of the fs geometry to
compute the minimum xfs log size correctly when formatting the
filesystem. This is the regression test for that issue.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
tests/xfs/761 | 45 +++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/761.out | 1 +
tests/xfs/group | 1 +
3 files changed, 47 insertions(+)
create mode 100755 tests/xfs/761
create mode 100644 tests/xfs/761.out
diff --git a/tests/xfs/761 b/tests/xfs/761
new file mode 100755
index 00000000..b9770d90
--- /dev/null
+++ b/tests/xfs/761
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 761
+#
+# Make sure mkfs sets up enough of the rt geometry that we can compute the
+# correct min log size for formatting the fs.
+#
+# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
+# required parts of the realtime geometry before computing log geometry").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_realtime
+
+rm -f $seqres.full
+
+# Format a tiny filesystem to force minimum log size, then see if it mounts
+_scratch_mkfs -r size=1m -d size=100m > $seqres.full
+_scratch_mount >> $seqres.full
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/761.out b/tests/xfs/761.out
new file mode 100644
index 00000000..8c9d9e90
--- /dev/null
+++ b/tests/xfs/761.out
@@ -0,0 +1 @@
+QA output created by 761
diff --git a/tests/xfs/group b/tests/xfs/group
index 318468b5..87badd56 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -503,4 +503,5 @@
758 auto quick rw attr realtime
759 auto quick rw realtime
760 auto quick rw collapse punch insert zero prealloc
+761 auto quick realtime
763 auto quick rw realtime
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes
2021-03-09 4:40 ` [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes Darrick J. Wong
@ 2021-03-11 9:32 ` Chandan Babu R
2021-03-14 14:51 ` Eryu Guan
1 sibling, 0 replies; 27+ messages in thread
From: Chandan Babu R @ 2021-03-11 9:32 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> In "mkfs: set required parts of the realtime geometry before computing
> log geometry" we made sure that mkfs set up enough of the fs geometry to
> compute the minimum xfs log size correctly when formatting the
> filesystem. This is the regression test for that issue.
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> tests/xfs/761 | 45 +++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/761.out | 1 +
> tests/xfs/group | 1 +
> 3 files changed, 47 insertions(+)
> create mode 100755 tests/xfs/761
> create mode 100644 tests/xfs/761.out
>
>
> diff --git a/tests/xfs/761 b/tests/xfs/761
> new file mode 100755
> index 00000000..b9770d90
> --- /dev/null
> +++ b/tests/xfs/761
> @@ -0,0 +1,45 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 761
> +#
> +# Make sure mkfs sets up enough of the rt geometry that we can compute the
> +# correct min log size for formatting the fs.
> +#
> +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
> +# required parts of the realtime geometry before computing log geometry").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_realtime
> +
> +rm -f $seqres.full
> +
> +# Format a tiny filesystem to force minimum log size, then see if it mounts
> +_scratch_mkfs -r size=1m -d size=100m > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/761.out b/tests/xfs/761.out
> new file mode 100644
> index 00000000..8c9d9e90
> --- /dev/null
> +++ b/tests/xfs/761.out
> @@ -0,0 +1 @@
> +QA output created by 761
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 318468b5..87badd56 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -503,4 +503,5 @@
> 758 auto quick rw attr realtime
> 759 auto quick rw realtime
> 760 auto quick rw collapse punch insert zero prealloc
> +761 auto quick realtime
Maybe "mount" should be added to the above list. Rest looks good to me.
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes
2021-03-09 4:40 ` [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes Darrick J. Wong
2021-03-11 9:32 ` Chandan Babu R
@ 2021-03-14 14:51 ` Eryu Guan
2021-03-14 16:39 ` Eryu Guan
1 sibling, 1 reply; 27+ messages in thread
From: Eryu Guan @ 2021-03-14 14:51 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests
On Mon, Mar 08, 2021 at 08:40:25PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> In "mkfs: set required parts of the realtime geometry before computing
> log geometry" we made sure that mkfs set up enough of the fs geometry to
> compute the minimum xfs log size correctly when formatting the
> filesystem. This is the regression test for that issue.
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> tests/xfs/761 | 45 +++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/761.out | 1 +
> tests/xfs/group | 1 +
> 3 files changed, 47 insertions(+)
> create mode 100755 tests/xfs/761
> create mode 100644 tests/xfs/761.out
>
>
> diff --git a/tests/xfs/761 b/tests/xfs/761
> new file mode 100755
> index 00000000..b9770d90
> --- /dev/null
> +++ b/tests/xfs/761
> @@ -0,0 +1,45 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 761
> +#
> +# Make sure mkfs sets up enough of the rt geometry that we can compute the
> +# correct min log size for formatting the fs.
> +#
> +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
> +# required parts of the realtime geometry before computing log geometry").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_realtime
> +
> +rm -f $seqres.full
> +
> +# Format a tiny filesystem to force minimum log size, then see if it mounts
> +_scratch_mkfs -r size=1m -d size=100m > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/761.out b/tests/xfs/761.out
> new file mode 100644
> index 00000000..8c9d9e90
> --- /dev/null
> +++ b/tests/xfs/761.out
> @@ -0,0 +1 @@
> +QA output created by 761
Need "Silence is golden" in .out file if you're going to respin it,
otherwise I can fix it on commit.
Thanks,
Eryu
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 318468b5..87badd56 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -503,4 +503,5 @@
> 758 auto quick rw attr realtime
> 759 auto quick rw realtime
> 760 auto quick rw collapse punch insert zero prealloc
> +761 auto quick realtime
> 763 auto quick rw realtime
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes
2021-03-14 14:51 ` Eryu Guan
@ 2021-03-14 16:39 ` Eryu Guan
0 siblings, 0 replies; 27+ messages in thread
From: Eryu Guan @ 2021-03-14 16:39 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests
On Sun, Mar 14, 2021 at 10:51:41PM +0800, Eryu Guan wrote:
> On Mon, Mar 08, 2021 at 08:40:25PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > In "mkfs: set required parts of the realtime geometry before computing
> > log geometry" we made sure that mkfs set up enough of the fs geometry to
> > compute the minimum xfs log size correctly when formatting the
> > filesystem. This is the regression test for that issue.
> >
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> > tests/xfs/761 | 45 +++++++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/761.out | 1 +
> > tests/xfs/group | 1 +
> > 3 files changed, 47 insertions(+)
> > create mode 100755 tests/xfs/761
> > create mode 100644 tests/xfs/761.out
> >
> >
> > diff --git a/tests/xfs/761 b/tests/xfs/761
> > new file mode 100755
> > index 00000000..b9770d90
> > --- /dev/null
> > +++ b/tests/xfs/761
> > @@ -0,0 +1,45 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle. All Rights Reserved.
> > +#
> > +# FS QA Test No. 761
> > +#
> > +# Make sure mkfs sets up enough of the rt geometry that we can compute the
> > +# correct min log size for formatting the fs.
> > +#
> > +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set
> > +# required parts of the realtime geometry before computing log geometry").
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_require_scratch
> > +_require_realtime
> > +
> > +rm -f $seqres.full
> > +
> > +# Format a tiny filesystem to force minimum log size, then see if it mounts
> > +_scratch_mkfs -r size=1m -d size=100m > $seqres.full
> > +_scratch_mount >> $seqres.full
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/xfs/761.out b/tests/xfs/761.out
> > new file mode 100644
> > index 00000000..8c9d9e90
> > --- /dev/null
> > +++ b/tests/xfs/761.out
> > @@ -0,0 +1 @@
> > +QA output created by 761
>
> Need "Silence is golden" in .out file if you're going to respin it,
> otherwise I can fix it on commit.
Never mind, I've applied it with .out file fixed, and added 'mount'
group as Chandan suggested.
Thanks,
Eryu
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 05/10] common/filter: refactor quota report filtering
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (3 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-11 9:42 ` Chandan Babu R
2021-03-09 4:40 ` [PATCH 06/10] xfs: test quota softlimit warning functionality Darrick J. Wong
` (5 subsequent siblings)
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
xfs/299 and xfs/050 share the same function to filter quota reporting
into a format suitable for the golden output. Refactor this so that we
can use it in a new test.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
common/filter | 24 ++++++++++++++++++++++++
tests/xfs/050 | 30 ++++++------------------------
tests/xfs/299 | 30 ++++++------------------------
3 files changed, 36 insertions(+), 48 deletions(-)
diff --git a/common/filter b/common/filter
index 2f3277f7..2efbbd99 100644
--- a/common/filter
+++ b/common/filter
@@ -637,5 +637,29 @@ _filter_getcap()
sed -e "s/= //" -e "s/\+/=/g"
}
+# Filter user/group/project id numbers out of quota reports, and standardize
+# the block counts to use filesystem block size. Callers must set the id and
+# bsize variables before calling this function.
+_filter_quota_report()
+{
+ test -n "$id" || echo "id must be set"
+ test -n "$bsize" || echo "block size must be set"
+
+ tr -s '[:space:]' | \
+ perl -npe '
+ s/^\#'$id' /[NAME] /g;
+ s/^\#0 \d+ /[ROOT] 0 /g;
+ s/6 days/7 days/g' |
+ perl -npe '
+ $val = 0;
+ if ($ENV{'LARGE_SCRATCH_DEV'}) {
+ $val = $ENV{'NUM_SPACE_FILES'};
+ }
+ s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
+ sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
+ perl -npe '
+ s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
+}
+
# make sure this script returns success
/bin/true
diff --git a/tests/xfs/050 b/tests/xfs/050
index 53412a13..1df97537 100755
--- a/tests/xfs/050
+++ b/tests/xfs/050
@@ -47,24 +47,6 @@ bhard=$(( 1000 * $bsize ))
isoft=4
ihard=10
-_filter_report()
-{
- tr -s '[:space:]' | \
- perl -npe '
- s/^\#'$id' /[NAME] /g;
- s/^\#0 \d+ /[ROOT] 0 /g;
- s/6 days/7 days/g' |
- perl -npe '
- $val = 0;
- if ($ENV{'LARGE_SCRATCH_DEV'}) {
- $val = $ENV{'NUM_SPACE_FILES'};
- }
- s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
- sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
- perl -npe '
- s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
-}
-
# The actual point at which limit enforcement takes place for the
# hard block limit is variable depending on filesystem blocksize,
# and iosize. What we want to test is that the limit is enforced
@@ -84,7 +66,7 @@ _filter_and_check_blks()
}
s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
}
- ' | _filter_report
+ ' | _filter_quota_report
}
_qsetup()
@@ -134,7 +116,7 @@ _exercise()
echo "*** report no quota settings" | tee -a $seqres.full
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
echo "*** report initial settings" | tee -a $seqres.full
@@ -147,7 +129,7 @@ _exercise()
$SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
echo "*** push past the soft inode limit" | tee -a $seqres.full
@@ -159,7 +141,7 @@ _exercise()
$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
echo "*** push past the soft block limit" | tee -a $seqres.full
@@ -169,7 +151,7 @@ _exercise()
-c "warn -b -$type 0 $id" $SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
# Note: for quota accounting (not enforcement), EDQUOT is not expected
@@ -183,7 +165,7 @@ _exercise()
-c "warn -i -$type 0 $id" $SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
# Note: for quota accounting (not enforcement), EDQUOT is not expected
diff --git a/tests/xfs/299 b/tests/xfs/299
index 15e0edf6..b862e67e 100755
--- a/tests/xfs/299
+++ b/tests/xfs/299
@@ -40,24 +40,6 @@ _require_xfs_quota
_require_xfs_mkfs_crc
_require_xfs_crc
-_filter_report()
-{
- tr -s '[:space:]' | \
- perl -npe '
- s/^\#'$id' /[NAME] /g;
- s/^\#0 \d+ /[ROOT] 0 /g;
- s/6 days/7 days/g' |
- perl -npe '
- $val = 0;
- if ($ENV{'LARGE_SCRATCH_DEV'}) {
- $val = $ENV{'NUM_SPACE_FILES'};
- }
- s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
- sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
- perl -npe '
- s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
-}
-
# The actual point at which limit enforcement takes place for the
# hard block limit is variable depending on filesystem blocksize,
# and iosize. What we want to test is that the limit is enforced
@@ -77,7 +59,7 @@ _filter_and_check_blks()
}
s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
}
- ' | _filter_report
+ ' | _filter_quota_report
}
_qsetup()
@@ -120,7 +102,7 @@ _exercise()
echo "*** report no quota settings" | tee -a $seqres.full
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
echo "*** report initial settings" | tee -a $seqres.full
@@ -133,7 +115,7 @@ _exercise()
$SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
echo "*** push past the soft inode limit" | tee -a $seqres.full
@@ -145,7 +127,7 @@ _exercise()
$XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
echo "*** push past the soft block limit" | tee -a $seqres.full
@@ -155,7 +137,7 @@ _exercise()
-c "warn -b -$type 0 $id" $SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
# Note: for quota accounting (not enforcement), EDQUOT is not expected
@@ -169,7 +151,7 @@ _exercise()
-c "warn -i -$type 0 $id" $SCRATCH_DEV
$XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
-c "repquota -birnN -$type" $SCRATCH_DEV |
- _filter_report | LC_COLLATE=POSIX sort -ru
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
echo
# Note: for quota accounting (not enforcement), EDQUOT is not expected
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 05/10] common/filter: refactor quota report filtering
2021-03-09 4:40 ` [PATCH 05/10] common/filter: refactor quota report filtering Darrick J. Wong
@ 2021-03-11 9:42 ` Chandan Babu R
0 siblings, 0 replies; 27+ messages in thread
From: Chandan Babu R @ 2021-03-11 9:42 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> xfs/299 and xfs/050 share the same function to filter quota reporting
> into a format suitable for the golden output. Refactor this so that we
> can use it in a new test.
>
Looks good to me.
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> common/filter | 24 ++++++++++++++++++++++++
> tests/xfs/050 | 30 ++++++------------------------
> tests/xfs/299 | 30 ++++++------------------------
> 3 files changed, 36 insertions(+), 48 deletions(-)
>
>
> diff --git a/common/filter b/common/filter
> index 2f3277f7..2efbbd99 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -637,5 +637,29 @@ _filter_getcap()
> sed -e "s/= //" -e "s/\+/=/g"
> }
>
> +# Filter user/group/project id numbers out of quota reports, and standardize
> +# the block counts to use filesystem block size. Callers must set the id and
> +# bsize variables before calling this function.
> +_filter_quota_report()
> +{
> + test -n "$id" || echo "id must be set"
> + test -n "$bsize" || echo "block size must be set"
> +
> + tr -s '[:space:]' | \
> + perl -npe '
> + s/^\#'$id' /[NAME] /g;
> + s/^\#0 \d+ /[ROOT] 0 /g;
> + s/6 days/7 days/g' |
> + perl -npe '
> + $val = 0;
> + if ($ENV{'LARGE_SCRATCH_DEV'}) {
> + $val = $ENV{'NUM_SPACE_FILES'};
> + }
> + s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
> + sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
> + perl -npe '
> + s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
> +}
> +
> # make sure this script returns success
> /bin/true
> diff --git a/tests/xfs/050 b/tests/xfs/050
> index 53412a13..1df97537 100755
> --- a/tests/xfs/050
> +++ b/tests/xfs/050
> @@ -47,24 +47,6 @@ bhard=$(( 1000 * $bsize ))
> isoft=4
> ihard=10
>
> -_filter_report()
> -{
> - tr -s '[:space:]' | \
> - perl -npe '
> - s/^\#'$id' /[NAME] /g;
> - s/^\#0 \d+ /[ROOT] 0 /g;
> - s/6 days/7 days/g' |
> - perl -npe '
> - $val = 0;
> - if ($ENV{'LARGE_SCRATCH_DEV'}) {
> - $val = $ENV{'NUM_SPACE_FILES'};
> - }
> - s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
> - sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
> - perl -npe '
> - s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
> -}
> -
> # The actual point at which limit enforcement takes place for the
> # hard block limit is variable depending on filesystem blocksize,
> # and iosize. What we want to test is that the limit is enforced
> @@ -84,7 +66,7 @@ _filter_and_check_blks()
> }
> s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
> }
> - ' | _filter_report
> + ' | _filter_quota_report
> }
>
> _qsetup()
> @@ -134,7 +116,7 @@ _exercise()
> echo "*** report no quota settings" | tee -a $seqres.full
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> echo "*** report initial settings" | tee -a $seqres.full
> @@ -147,7 +129,7 @@ _exercise()
> $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> echo "*** push past the soft inode limit" | tee -a $seqres.full
> @@ -159,7 +141,7 @@ _exercise()
> $XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> echo "*** push past the soft block limit" | tee -a $seqres.full
> @@ -169,7 +151,7 @@ _exercise()
> -c "warn -b -$type 0 $id" $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> # Note: for quota accounting (not enforcement), EDQUOT is not expected
> @@ -183,7 +165,7 @@ _exercise()
> -c "warn -i -$type 0 $id" $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> # Note: for quota accounting (not enforcement), EDQUOT is not expected
> diff --git a/tests/xfs/299 b/tests/xfs/299
> index 15e0edf6..b862e67e 100755
> --- a/tests/xfs/299
> +++ b/tests/xfs/299
> @@ -40,24 +40,6 @@ _require_xfs_quota
> _require_xfs_mkfs_crc
> _require_xfs_crc
>
> -_filter_report()
> -{
> - tr -s '[:space:]' | \
> - perl -npe '
> - s/^\#'$id' /[NAME] /g;
> - s/^\#0 \d+ /[ROOT] 0 /g;
> - s/6 days/7 days/g' |
> - perl -npe '
> - $val = 0;
> - if ($ENV{'LARGE_SCRATCH_DEV'}) {
> - $val = $ENV{'NUM_SPACE_FILES'};
> - }
> - s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' |
> - sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' |
> - perl -npe '
> - s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|'
> -}
> -
> # The actual point at which limit enforcement takes place for the
> # hard block limit is variable depending on filesystem blocksize,
> # and iosize. What we want to test is that the limit is enforced
> @@ -77,7 +59,7 @@ _filter_and_check_blks()
> }
> s/^(\#'$id'\s+)(\d+)/\1 =OK=/g;
> }
> - ' | _filter_report
> + ' | _filter_quota_report
> }
>
> _qsetup()
> @@ -120,7 +102,7 @@ _exercise()
> echo "*** report no quota settings" | tee -a $seqres.full
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> echo "*** report initial settings" | tee -a $seqres.full
> @@ -133,7 +115,7 @@ _exercise()
> $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> echo "*** push past the soft inode limit" | tee -a $seqres.full
> @@ -145,7 +127,7 @@ _exercise()
> $XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> echo "*** push past the soft block limit" | tee -a $seqres.full
> @@ -155,7 +137,7 @@ _exercise()
> -c "warn -b -$type 0 $id" $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> # Note: for quota accounting (not enforcement), EDQUOT is not expected
> @@ -169,7 +151,7 @@ _exercise()
> -c "warn -i -$type 0 $id" $SCRATCH_DEV
> $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> -c "repquota -birnN -$type" $SCRATCH_DEV |
> - _filter_report | LC_COLLATE=POSIX sort -ru
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
>
> echo
> # Note: for quota accounting (not enforcement), EDQUOT is not expected
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 06/10] xfs: test quota softlimit warning functionality
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (4 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 05/10] common/filter: refactor quota report filtering Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-11 11:10 ` Chandan Babu R
2021-03-09 4:40 ` [PATCH 07/10] xfs/122: fix test for xfs_attr_shortform_t conversion Darrick J. Wong
` (4 subsequent siblings)
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
Make sure that quota softlimits work, which is to say that one can
exceed the softlimit up to warnlimit times before it starts enforcing
that.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
tests/xfs/915 | 162 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/915.out | 151 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/group | 1
3 files changed, 314 insertions(+)
create mode 100755 tests/xfs/915
create mode 100644 tests/xfs/915.out
diff --git a/tests/xfs/915 b/tests/xfs/915
new file mode 100755
index 00000000..a2cdbbb7
--- /dev/null
+++ b/tests/xfs/915
@@ -0,0 +1,162 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 915
+#
+# Check that quota softlimit warnings work the way they should. This means
+# that we can disobey the softlimit up to warnlimit times before it turns into
+# hard(er) enforcement. This is a functional test for quota warnings, but
+# since the functionality has been broken for decades, this is also a
+# regression test for commit 4b8628d57b72 ("xfs: actually bump warning counts
+# when we send warnings").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/quota
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_quota
+_require_scratch
+
+rm -f $seqres.full
+
+qsetup()
+{
+ opt=$1
+ enforce=0
+ if [ $opt = "u" -o $opt = "uno" ]; then
+ type=u
+ eval `_choose_uid`
+ elif [ $opt = "g" -o $opt = "gno" ]; then
+ type=g
+ eval `_choose_gid`
+ elif [ $opt = "p" -o $opt = "pno" ]; then
+ type=p
+ eval `_choose_prid`
+ fi
+ [ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
+
+ echo "Using type=$type id=$id" >> $seqres.full
+}
+
+exercise() {
+ _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
+ cat $tmp.mkfs >>$seqres.full
+
+ # keep the blocksize and data size for dd later
+ . $tmp.mkfs
+
+ _qmount
+
+ qsetup $1
+
+ echo "Using type=$type id=$id" >>$seqres.full
+
+ echo
+ echo "*** report initial settings" | tee -a $seqres.full
+ $XFS_QUOTA_PROG -x \
+ -c "limit -$type isoft=3 ihard=500000 $id" \
+ -c "warn -$type -i -d 13" \
+ $SCRATCH_DEV
+ $XFS_QUOTA_PROG -x \
+ -c "state -$type" >> $seqres.full
+ $XFS_QUOTA_PROG -x \
+ -c "repquota -birnN -$type" $SCRATCH_DEV |
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+ echo
+ echo "*** push past the soft inode limit" | tee -a $seqres.full
+ _file_as_id $SCRATCH_MNT/softok1 $id $type $bsize 0
+ _file_as_id $SCRATCH_MNT/softok2 $id $type $bsize 0
+ _file_as_id $SCRATCH_MNT/softok3 $id $type $bsize 0
+ _file_as_id $SCRATCH_MNT/softwarn1 $id $type $bsize 0
+ $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+ -c "repquota -birnN -$type" $SCRATCH_DEV |
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+ echo
+ echo "*** push further past the soft inode limit" | tee -a $seqres.full
+ for warn_nr in $(seq 2 5); do
+ _file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
+ done
+ $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+ -c "repquota -birnN -$type" $SCRATCH_DEV |
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+ echo
+ echo "*** push past the soft inode warning limit" | tee -a $seqres.full
+ for warn_nr in $(seq 6 15); do
+ _file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
+ done
+ $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
+ -c "repquota -birnN -$type" $SCRATCH_DEV |
+ _filter_quota_report | LC_COLLATE=POSIX sort -ru
+
+ echo
+ echo "*** unmount"
+ _scratch_unmount
+}
+
+_scratch_mkfs > $seqres.full
+_scratch_mount >> $seqres.full
+
+chmod a+rwx $SCRATCH_MNT $seqres.full # arbitrary users will write here
+bsize=$(_get_file_block_size $SCRATCH_MNT)
+_scratch_unmount
+
+cat >$tmp.projects <<EOF
+1:$SCRATCH_MNT
+EOF
+
+cat >$tmp.projid <<EOF
+root:0
+scratch:1
+EOF
+
+projid_file="$tmp.projid"
+
+echo "*** user"
+_qmount_option "uquota"
+exercise u
+
+echo "*** group"
+_qmount_option "gquota"
+exercise g
+
+echo "*** uqnoenforce"
+_qmount_option "uqnoenforce"
+exercise uno
+
+echo "*** gqnoenforce"
+_qmount_option "gqnoenforce"
+exercise gno
+
+echo "*** pquota"
+_qmount_option "pquota"
+exercise p
+
+echo "*** pqnoenforce"
+_qmount_option "pqnoenforce"
+exercise pno
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/915.out b/tests/xfs/915.out
new file mode 100644
index 00000000..c3bb855e
--- /dev/null
+++ b/tests/xfs/915.out
@@ -0,0 +1,151 @@
+QA output created by 915
+*** user
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
+
+*** unmount
+*** group
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
+
+*** unmount
+*** uqnoenforce
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** unmount
+*** gqnoenforce
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** unmount
+*** pquota
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 02 [7 days] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 06 [7 days] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 15 3 500000 13 [7 days] 0 0 0 00 [--------]
+
+*** unmount
+*** pqnoenforce
+meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
+data = bsize=XXX blocks=XXX, imaxpct=PCT
+ = sunit=XXX swidth=XXX, unwritten=X
+naming =VERN bsize=XXX
+log =LDEV bsize=XXX blocks=XXX
+realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
+
+*** report initial settings
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push further past the soft inode limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** push past the soft inode warning limit
+[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
+[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
+
+*** unmount
diff --git a/tests/xfs/group b/tests/xfs/group
index 87badd56..d7aafc94 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -505,3 +505,4 @@
760 auto quick rw collapse punch insert zero prealloc
761 auto quick realtime
763 auto quick rw realtime
+915 auto quick quota
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 06/10] xfs: test quota softlimit warning functionality
2021-03-09 4:40 ` [PATCH 06/10] xfs: test quota softlimit warning functionality Darrick J. Wong
@ 2021-03-11 11:10 ` Chandan Babu R
2021-03-12 4:25 ` Chandan Babu R
0 siblings, 1 reply; 27+ messages in thread
From: Chandan Babu R @ 2021-03-11 11:10 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Make sure that quota softlimits work, which is to say that one can
> exceed the softlimit up to warnlimit times before it starts enforcing
> that.
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> tests/xfs/915 | 162 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/915.out | 151 +++++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/group | 1
> 3 files changed, 314 insertions(+)
> create mode 100755 tests/xfs/915
> create mode 100644 tests/xfs/915.out
>
>
> diff --git a/tests/xfs/915 b/tests/xfs/915
> new file mode 100755
> index 00000000..a2cdbbb7
> --- /dev/null
> +++ b/tests/xfs/915
> @@ -0,0 +1,162 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 915
> +#
> +# Check that quota softlimit warnings work the way they should. This means
> +# that we can disobey the softlimit up to warnlimit times before it turns into
> +# hard(er) enforcement. This is a functional test for quota warnings, but
> +# since the functionality has been broken for decades, this is also a
> +# regression test for commit 4b8628d57b72 ("xfs: actually bump warning counts
> +# when we send warnings").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/quota
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_xfs_quota
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +qsetup()
> +{
> + opt=$1
> + enforce=0
> + if [ $opt = "u" -o $opt = "uno" ]; then
> + type=u
> + eval `_choose_uid`
> + elif [ $opt = "g" -o $opt = "gno" ]; then
> + type=g
> + eval `_choose_gid`
> + elif [ $opt = "p" -o $opt = "pno" ]; then
> + type=p
> + eval `_choose_prid`
> + fi
> + [ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1
> +
> + echo "Using type=$type id=$id" >> $seqres.full
> +}
> +
> +exercise() {
> + _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
> + cat $tmp.mkfs >>$seqres.full
> +
> + # keep the blocksize and data size for dd later
> + . $tmp.mkfs
> +
> + _qmount
> +
> + qsetup $1
> +
> + echo "Using type=$type id=$id" >>$seqres.full
> +
> + echo
> + echo "*** report initial settings" | tee -a $seqres.full
> + $XFS_QUOTA_PROG -x \
> + -c "limit -$type isoft=3 ihard=500000 $id" \
> + -c "warn -$type -i -d 13" \
> + $SCRATCH_DEV
> + $XFS_QUOTA_PROG -x \
> + -c "state -$type" >> $seqres.full
> + $XFS_QUOTA_PROG -x \
> + -c "repquota -birnN -$type" $SCRATCH_DEV |
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> + echo
> + echo "*** push past the soft inode limit" | tee -a $seqres.full
> + _file_as_id $SCRATCH_MNT/softok1 $id $type $bsize 0
> + _file_as_id $SCRATCH_MNT/softok2 $id $type $bsize 0
> + _file_as_id $SCRATCH_MNT/softok3 $id $type $bsize 0
> + _file_as_id $SCRATCH_MNT/softwarn1 $id $type $bsize 0
> + $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> + -c "repquota -birnN -$type" $SCRATCH_DEV |
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> + echo
> + echo "*** push further past the soft inode limit" | tee -a $seqres.full
> + for warn_nr in $(seq 2 5); do
> + _file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
> + done
> + $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> + -c "repquota -birnN -$type" $SCRATCH_DEV |
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> + echo
> + echo "*** push past the soft inode warning limit" | tee -a $seqres.full
> + for warn_nr in $(seq 6 15); do
> + _file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0
> + done
> + $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \
> + -c "repquota -birnN -$type" $SCRATCH_DEV |
> + _filter_quota_report | LC_COLLATE=POSIX sort -ru
> +
> + echo
> + echo "*** unmount"
> + _scratch_unmount
> +}
> +
> +_scratch_mkfs > $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +chmod a+rwx $SCRATCH_MNT $seqres.full # arbitrary users will write here
> +bsize=$(_get_file_block_size $SCRATCH_MNT)
> +_scratch_unmount
> +
> +cat >$tmp.projects <<EOF
> +1:$SCRATCH_MNT
> +EOF
> +
> +cat >$tmp.projid <<EOF
> +root:0
> +scratch:1
> +EOF
> +
> +projid_file="$tmp.projid"
> +
> +echo "*** user"
> +_qmount_option "uquota"
> +exercise u
> +
> +echo "*** group"
> +_qmount_option "gquota"
> +exercise g
> +
> +echo "*** uqnoenforce"
> +_qmount_option "uqnoenforce"
> +exercise uno
> +
> +echo "*** gqnoenforce"
> +_qmount_option "gqnoenforce"
> +exercise gno
> +
> +echo "*** pquota"
> +_qmount_option "pquota"
> +exercise p
> +
> +echo "*** pqnoenforce"
> +_qmount_option "pqnoenforce"
> +exercise pno
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/915.out b/tests/xfs/915.out
> new file mode 100644
> index 00000000..c3bb855e
> --- /dev/null
> +++ b/tests/xfs/915.out
> @@ -0,0 +1,151 @@
> +QA output created by 915
> +*** user
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** group
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 01 [7 days] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 05 [7 days] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 16 3 500000 13 [7 days] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** uqnoenforce
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** gqnoenforce
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push further past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 8 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode warning limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 18 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** unmount
> +*** pquota
> +meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
> +data = bsize=XXX blocks=XXX, imaxpct=PCT
> + = sunit=XXX swidth=XXX, unwritten=X
> +naming =VERN bsize=XXX
> +log =LDEV bsize=XXX blocks=XXX
> +realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> +
> +*** report initial settings
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
> +
> +*** push past the soft inode limit
> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
> +[NAME] 0 0 0 00 [--------] 4 3 500000 02 [7 days] 0 0 0 00 [--------]
At this point in the test we have created 4 files.
1. softok{1,2,3}
2. softwarn1
So we have exceeded the soft inode limit (i.e. 3) once. But the warning has
been issued twice.
_file_as_id() changes the project id of parent of each of the above files. In
this case all the above listed files have $SCRATCH_MNT as the parent. So by
the time softok2 is created we have already reached the soft inode limit of 3
(parent and the two softok{1,2} files) and creation of softok3 and softwarn1
generates the two warnings listed above. If this explaination is correct,
shouldn't 'Used' inode count have a value of 5 (including the inode associated
with $SCRATCH_MNT)?
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 06/10] xfs: test quota softlimit warning functionality
2021-03-11 11:10 ` Chandan Babu R
@ 2021-03-12 4:25 ` Chandan Babu R
0 siblings, 0 replies; 27+ messages in thread
From: Chandan Babu R @ 2021-03-12 4:25 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 11 Mar 2021 at 16:40, Chandan Babu R wrote:
> On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
>> From: Darrick J. Wong <djwong@kernel.org>
>>
>> Make sure that quota softlimits work, which is to say that one can
>> exceed the softlimit up to warnlimit times before it starts enforcing
>> that.
>>
>> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>> ---
>> tests/xfs/915 | 162 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/xfs/915.out | 151 +++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/xfs/group | 1
>> 3 files changed, 314 insertions(+)
>> create mode 100755 tests/xfs/915
>> create mode 100644 tests/xfs/915.out
>>
>>
>> diff --git a/tests/xfs/915 b/tests/xfs/915
[...]
>> +*** report initial settings
>> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
>> +[NAME] 0 0 0 00 [--------] 0 3 500000 00 [--------] 0 0 0 00 [--------]
>> +
>> +*** push past the soft inode limit
>> +[ROOT] 0 0 0 00 [--------] 3 0 0 13 [--------] 0 0 0 00 [--------]
>> +[NAME] 0 0 0 00 [--------] 4 3 500000 02 [7 days] 0 0 0 00 [--------]
>
> At this point in the test we have created 4 files.
> 1. softok{1,2,3}
> 2. softwarn1
>
> So we have exceeded the soft inode limit (i.e. 3) once. But the warning has
> been issued twice.
>
> _file_as_id() changes the project id of parent of each of the above files. In
> this case all the above listed files have $SCRATCH_MNT as the parent. So by
> the time softok2 is created we have already reached the soft inode limit of 3
> (parent and the two softok{1,2} files) and creation of softok3 and softwarn1
> generates the two warnings listed above. If this explaination is correct,
> shouldn't 'Used' inode count have a value of 5 (including the inode associated
> with $SCRATCH_MNT)?
Sorry, I missed the fact that _file_as_id() reverts back the project id of
$SCRATCH_MNT just before returning.
The patch looks good.
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 07/10] xfs/122: fix test for xfs_attr_shortform_t conversion
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (5 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 06/10] xfs: test quota softlimit warning functionality Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-11 12:58 ` Christoph Hellwig
2021-03-09 4:40 ` [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE Darrick J. Wong
` (3 subsequent siblings)
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: Chandan Babu R, linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
The typedef xfs_attr_shortform_t was converted to a struct in 5.10.
Update this test to pass.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
---
tests/xfs/122 | 1 -
tests/xfs/122.out | 1 +
2 files changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/xfs/122 b/tests/xfs/122
index a4248031..322e1d81 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -72,7 +72,6 @@ for hdr in /usr/include/xfs/xfs*.h; do
done
# missing:
-# xfs_attr_shortform_t
# xfs_trans_header_t
cat >$tmp.ignore <<EOF
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 45c42e59..cfe09c6d 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -62,6 +62,7 @@ sizeof(struct xfs_agfl) = 36
sizeof(struct xfs_attr3_leaf_hdr) = 80
sizeof(struct xfs_attr3_leafblock) = 88
sizeof(struct xfs_attr3_rmt_hdr) = 56
+sizeof(struct xfs_attr_shortform) = 8
sizeof(struct xfs_btree_block) = 72
sizeof(struct xfs_btree_block_lhdr) = 64
sizeof(struct xfs_btree_block_shdr) = 48
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (6 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 07/10] xfs/122: fix test for xfs_attr_shortform_t conversion Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-12 7:13 ` Chandan Babu R
2021-03-09 4:40 ` [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files Darrick J. Wong
` (2 subsequent siblings)
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
Create a test to make sure that dedupe actually locks the file ranges
correctly before starting the content comparison and keeps them locked
until the operation completes.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
src/Makefile | 2
src/deduperace.c | 370 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/generic/949 | 51 +++++++
tests/generic/949.out | 2
tests/generic/group | 1
5 files changed, 425 insertions(+), 1 deletion(-)
create mode 100644 src/deduperace.c
create mode 100755 tests/generic/949
create mode 100644 tests/generic/949.out
diff --git a/src/Makefile b/src/Makefile
index 811b24e4..38ee6718 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -21,7 +21,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \
LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \
- locktest unwritten_mmap bulkstat_unlink_test \
+ locktest unwritten_mmap bulkstat_unlink_test deduperace \
bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \
diff --git a/src/deduperace.c b/src/deduperace.c
new file mode 100644
index 00000000..b252d436
--- /dev/null
+++ b/src/deduperace.c
@@ -0,0 +1,370 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2021 Oracle. All Rights Reserved.
+ * Author: Darrick J. Wong <djwong@kernel.org>
+ *
+ * Race pwrite/mwrite with dedupe to see if we got the locking right.
+ *
+ * File writes and mmap writes should not be able to change the src_fd's
+ * contents after dedupe prep has verified that the file contents are the same.
+ */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+#include <sys/ioctl.h>
+#include <linux/fs.h>
+#include <string.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdlib.h>
+#include <errno.h>
+
+#define GOOD_BYTE 0x58
+#define BAD_BYTE 0x66
+
+#ifndef FIDEDUPERANGE
+/* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */
+#define FILE_DEDUPE_RANGE_SAME 0
+#define FILE_DEDUPE_RANGE_DIFFERS 1
+
+/* from struct btrfs_ioctl_file_extent_same_info */
+struct file_dedupe_range_info {
+ __s64 dest_fd; /* in - destination file */
+ __u64 dest_offset; /* in - start of extent in destination */
+ __u64 bytes_deduped; /* out - total # of bytes we were able
+ * to dedupe from this file. */
+ /* status of this dedupe operation:
+ * < 0 for error
+ * == FILE_DEDUPE_RANGE_SAME if dedupe succeeds
+ * == FILE_DEDUPE_RANGE_DIFFERS if data differs
+ */
+ __s32 status; /* out - see above description */
+ __u32 reserved; /* must be zero */
+};
+
+/* from struct btrfs_ioctl_file_extent_same_args */
+struct file_dedupe_range {
+ __u64 src_offset; /* in - start of extent in source */
+ __u64 src_length; /* in - length of extent */
+ __u16 dest_count; /* in - total elements in info array */
+ __u16 reserved1; /* must be zero */
+ __u32 reserved2; /* must be zero */
+ struct file_dedupe_range_info info[0];
+};
+#define FIDEDUPERANGE _IOWR(0x94, 54, struct file_dedupe_range)
+#endif /* FIDEDUPERANGE */
+
+static int fd1, fd2;
+static loff_t offset = 37; /* Nice low offset to trick the compare */
+static loff_t blksz;
+
+/* Continuously dirty the pagecache for the region being dupe-tested. */
+void *
+mwriter(
+ void *data)
+{
+ volatile char *p;
+
+ p = mmap(NULL, blksz, PROT_WRITE, MAP_SHARED, fd1, 0);
+ if (p == MAP_FAILED) {
+ perror("mmap");
+ exit(2);
+ }
+
+ while (1) {
+ *(p + offset) = BAD_BYTE;
+ *(p + offset) = GOOD_BYTE;
+ }
+}
+
+/* Continuously write to the region being dupe-tested. */
+void *
+pwriter(
+ void *data)
+{
+ char v;
+ ssize_t sz;
+
+ while (1) {
+ v = BAD_BYTE;
+ sz = pwrite(fd1, &v, sizeof(v), offset);
+ if (sz != sizeof(v)) {
+ perror("pwrite0");
+ exit(2);
+ }
+
+ v = GOOD_BYTE;
+ sz = pwrite(fd1, &v, sizeof(v), offset);
+ if (sz != sizeof(v)) {
+ perror("pwrite1");
+ exit(2);
+ }
+ }
+
+ return NULL;
+}
+
+static inline void
+complain(
+ loff_t offset,
+ char bad)
+{
+ fprintf(stderr, "ASSERT: offset %llu should be 0x%x, got 0x%x!\n",
+ (unsigned long long)offset, GOOD_BYTE, bad);
+ abort();
+}
+
+/* Make sure the destination file pagecache never changes. */
+void *
+mreader(
+ void *data)
+{
+ volatile char *p;
+
+ p = mmap(NULL, blksz, PROT_READ, MAP_SHARED, fd2, 0);
+ if (p == MAP_FAILED) {
+ perror("mmap");
+ exit(2);
+ }
+
+ while (1) {
+ if (*(p + offset) != GOOD_BYTE)
+ complain(offset, *(p + offset));
+ }
+}
+
+/* Make sure the destination file never changes. */
+void *
+preader(
+ void *data)
+{
+ char v;
+ ssize_t sz;
+
+ while (1) {
+ sz = pread(fd2, &v, sizeof(v), offset);
+ if (sz != sizeof(v)) {
+ perror("pwrite0");
+ exit(2);
+ }
+
+ if (v != GOOD_BYTE)
+ complain(offset, v);
+ }
+
+ return NULL;
+}
+
+void
+print_help(const char *progname)
+{
+ printf("Usage: %s [-b blksz] [-c dir] [-n nr_ops] [-o offset] [-r] [-w] [-v]\n",
+ progname);
+ printf("-b sets the block size (default is autoconfigured)\n");
+ printf("-c chdir to this path before starting\n");
+ printf("-n controls the number of dedupe ops (default 10000)\n");
+ printf("-o reads and writes to this offset (default 37)\n");
+ printf("-r uses pread instead of mmap read.\n");
+ printf("-v prints status updates.\n");
+ printf("-w uses pwrite instead of mmap write.\n");
+}
+
+int
+main(
+ int argc,
+ char *argv[])
+{
+ struct file_dedupe_range *fdr;
+ char *Xbuf;
+ void *(*reader_fn)(void *) = mreader;
+ void *(*writer_fn)(void *) = mwriter;
+ unsigned long same = 0;
+ unsigned long differs = 0;
+ unsigned long i, nr_ops = 10000;
+ ssize_t sz;
+ pthread_t reader, writer;
+ int verbose = 0;
+ int c;
+ int ret;
+
+ while ((c = getopt(argc, argv, "b:c:n:o:rvw")) != -1) {
+ switch (c) {
+ case 'b':
+ errno = 0;
+ blksz = strtoul(optarg, NULL, 0);
+ if (errno) {
+ perror(optarg);
+ exit(1);
+ }
+ break;
+ case 'c':
+ ret = chdir(optarg);
+ if (ret) {
+ perror("chdir");
+ exit(1);
+ }
+ break;
+ case 'n':
+ errno = 0;
+ nr_ops = strtoul(optarg, NULL, 0);
+ if (errno) {
+ perror(optarg);
+ exit(1);
+ }
+ break;
+ case 'o':
+ errno = 0;
+ offset = strtoul(optarg, NULL, 0);
+ if (errno) {
+ perror(optarg);
+ exit(1);
+ }
+ break;
+ case 'r':
+ reader_fn = preader;
+ break;
+ case 'v':
+ verbose = 1;
+ break;
+ case 'w':
+ writer_fn = pwriter;
+ break;
+ default:
+ print_help(argv[0]);
+ exit(1);
+ break;
+ }
+ }
+
+ fdr = malloc(sizeof(struct file_dedupe_range) +
+ sizeof(struct file_dedupe_range_info));
+ if (!fdr) {
+ perror("malloc");
+ exit(1);
+ }
+
+ /* Initialize both files. */
+ fd1 = open("file1", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
+ if (fd1 < 0) {
+ perror("file1");
+ exit(1);
+ }
+
+ fd2 = open("file2", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
+ if (fd2 < 0) {
+ perror("file2");
+ exit(1);
+ }
+
+ if (blksz <= 0) {
+ struct stat statbuf;
+
+ ret = fstat(fd1, &statbuf);
+ if (ret) {
+ perror("file1 stat");
+ exit(1);
+ }
+ blksz = statbuf.st_blksize;
+ }
+
+ if (offset >= blksz) {
+ fprintf(stderr, "offset (%llu) < blksz (%llu)?\n",
+ (unsigned long long)offset,
+ (unsigned long long)blksz);
+ exit(1);
+ }
+
+ Xbuf = malloc(blksz);
+ if (!Xbuf) {
+ perror("malloc buffer");
+ exit(1);
+ }
+ memset(Xbuf, GOOD_BYTE, blksz);
+
+ sz = pwrite(fd1, Xbuf, blksz, 0);
+ if (sz != blksz) {
+ perror("file1 write");
+ exit(1);
+ }
+
+ sz = pwrite(fd2, Xbuf, blksz, 0);
+ if (sz != blksz) {
+ perror("file2 write");
+ exit(1);
+ }
+
+ ret = fsync(fd1);
+ if (ret) {
+ perror("file1 fsync");
+ exit(1);
+ }
+
+ ret = fsync(fd2);
+ if (ret) {
+ perror("file2 fsync");
+ exit(1);
+ }
+
+ /* Start our reader and writer threads. */
+ ret = pthread_create(&reader, NULL, reader_fn, NULL);
+ if (ret) {
+ fprintf(stderr, "rthread: %s\n", strerror(ret));
+ exit(1);
+ }
+
+ ret = pthread_create(&writer, NULL, writer_fn, NULL);
+ if (ret) {
+ fprintf(stderr, "wthread: %s\n", strerror(ret));
+ exit(1);
+ }
+
+ /*
+ * Now start deduping. If the contents match, fd1's blocks will be
+ * remapped into fd2, which is why the writer thread targets fd1 and
+ * the reader checks fd2 to make sure that none of fd1's writes ever
+ * make it into fd2.
+ */
+ for (i = 1; i <= nr_ops; i++) {
+ fdr->src_offset = 0;
+ fdr->src_length = blksz;
+ fdr->dest_count = 1;
+ fdr->reserved1 = 0;
+ fdr->reserved2 = 0;
+ fdr->info[0].dest_fd = fd2;
+ fdr->info[0].dest_offset = 0;
+ fdr->info[0].reserved = 0;
+
+ ret = ioctl(fd1, FIDEDUPERANGE, fdr);
+ if (ret) {
+ perror("dedupe");
+ exit(2);
+ }
+
+ switch (fdr->info[0].status) {
+ case FILE_DEDUPE_RANGE_DIFFERS:
+ differs++;
+ break;
+ case FILE_DEDUPE_RANGE_SAME:
+ same++;
+ break;
+ default:
+ fprintf(stderr, "deduperange: %s\n",
+ strerror(-fdr->info[0].status));
+ exit(2);
+ break;
+ }
+
+ if (verbose && (i % 337) == 0)
+ printf("nr_ops: %lu; same: %lu; differs: %lu\n",
+ i, same, differs);
+ }
+
+ if (verbose)
+ printf("nr_ops: %lu; same: %lu; differs: %lu\n", i - 1, same,
+ differs);
+
+ /* Program termination will kill the threads and close the files. */
+ return 0;
+}
diff --git a/tests/generic/949 b/tests/generic/949
new file mode 100755
index 00000000..3951490b
--- /dev/null
+++ b/tests/generic/949
@@ -0,0 +1,51 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 949
+#
+# Make sure that mmap and file writers racing with FIDEDUPERANGE cannot write
+# to the file after the dedupe prep function has decided that the file contents
+# are identical and we can therefore go ahead with the remapping.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch_dedupe
+
+rm -f $seqres.full
+
+nr_ops=$((TIME_FACTOR * 10000))
+
+# Format filesystem
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Test once with mmap writes
+$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops
+
+# Test again with pwrites for the lulz
+$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops -w
+
+echo Silence is golden.
+# success, all done
+status=0
+exit
diff --git a/tests/generic/949.out b/tests/generic/949.out
new file mode 100644
index 00000000..2998b46c
--- /dev/null
+++ b/tests/generic/949.out
@@ -0,0 +1,2 @@
+QA output created by 949
+Silence is golden.
diff --git a/tests/generic/group b/tests/generic/group
index d5cfdd51..778aa8c4 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -630,3 +630,4 @@
625 auto quick verity
947 auto quick rw clone
948 auto quick rw copy_range
+949 auto quick rw dedupe clone
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE
2021-03-09 4:40 ` [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE Darrick J. Wong
@ 2021-03-12 7:13 ` Chandan Babu R
0 siblings, 0 replies; 27+ messages in thread
From: Chandan Babu R @ 2021-03-12 7:13 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Create a test to make sure that dedupe actually locks the file ranges
> correctly before starting the content comparison and keeps them locked
> until the operation completes.
>
Looks good to me.
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> src/Makefile | 2
> src/deduperace.c | 370 +++++++++++++++++++++++++++++++++++++++++++++++++
> tests/generic/949 | 51 +++++++
> tests/generic/949.out | 2
> tests/generic/group | 1
> 5 files changed, 425 insertions(+), 1 deletion(-)
> create mode 100644 src/deduperace.c
> create mode 100755 tests/generic/949
> create mode 100644 tests/generic/949.out
>
>
> diff --git a/src/Makefile b/src/Makefile
> index 811b24e4..38ee6718 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -21,7 +21,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \
>
> LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
> preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \
> - locktest unwritten_mmap bulkstat_unlink_test \
> + locktest unwritten_mmap bulkstat_unlink_test deduperace \
> bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
> stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
> seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \
> diff --git a/src/deduperace.c b/src/deduperace.c
> new file mode 100644
> index 00000000..b252d436
> --- /dev/null
> +++ b/src/deduperace.c
> @@ -0,0 +1,370 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2021 Oracle. All Rights Reserved.
> + * Author: Darrick J. Wong <djwong@kernel.org>
> + *
> + * Race pwrite/mwrite with dedupe to see if we got the locking right.
> + *
> + * File writes and mmap writes should not be able to change the src_fd's
> + * contents after dedupe prep has verified that the file contents are the same.
> + */
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <sys/mman.h>
> +#include <sys/ioctl.h>
> +#include <linux/fs.h>
> +#include <string.h>
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +#include <pthread.h>
> +#include <stdlib.h>
> +#include <errno.h>
> +
> +#define GOOD_BYTE 0x58
> +#define BAD_BYTE 0x66
> +
> +#ifndef FIDEDUPERANGE
> +/* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */
> +#define FILE_DEDUPE_RANGE_SAME 0
> +#define FILE_DEDUPE_RANGE_DIFFERS 1
> +
> +/* from struct btrfs_ioctl_file_extent_same_info */
> +struct file_dedupe_range_info {
> + __s64 dest_fd; /* in - destination file */
> + __u64 dest_offset; /* in - start of extent in destination */
> + __u64 bytes_deduped; /* out - total # of bytes we were able
> + * to dedupe from this file. */
> + /* status of this dedupe operation:
> + * < 0 for error
> + * == FILE_DEDUPE_RANGE_SAME if dedupe succeeds
> + * == FILE_DEDUPE_RANGE_DIFFERS if data differs
> + */
> + __s32 status; /* out - see above description */
> + __u32 reserved; /* must be zero */
> +};
> +
> +/* from struct btrfs_ioctl_file_extent_same_args */
> +struct file_dedupe_range {
> + __u64 src_offset; /* in - start of extent in source */
> + __u64 src_length; /* in - length of extent */
> + __u16 dest_count; /* in - total elements in info array */
> + __u16 reserved1; /* must be zero */
> + __u32 reserved2; /* must be zero */
> + struct file_dedupe_range_info info[0];
> +};
> +#define FIDEDUPERANGE _IOWR(0x94, 54, struct file_dedupe_range)
> +#endif /* FIDEDUPERANGE */
> +
> +static int fd1, fd2;
> +static loff_t offset = 37; /* Nice low offset to trick the compare */
> +static loff_t blksz;
> +
> +/* Continuously dirty the pagecache for the region being dupe-tested. */
> +void *
> +mwriter(
> + void *data)
> +{
> + volatile char *p;
> +
> + p = mmap(NULL, blksz, PROT_WRITE, MAP_SHARED, fd1, 0);
> + if (p == MAP_FAILED) {
> + perror("mmap");
> + exit(2);
> + }
> +
> + while (1) {
> + *(p + offset) = BAD_BYTE;
> + *(p + offset) = GOOD_BYTE;
> + }
> +}
> +
> +/* Continuously write to the region being dupe-tested. */
> +void *
> +pwriter(
> + void *data)
> +{
> + char v;
> + ssize_t sz;
> +
> + while (1) {
> + v = BAD_BYTE;
> + sz = pwrite(fd1, &v, sizeof(v), offset);
> + if (sz != sizeof(v)) {
> + perror("pwrite0");
> + exit(2);
> + }
> +
> + v = GOOD_BYTE;
> + sz = pwrite(fd1, &v, sizeof(v), offset);
> + if (sz != sizeof(v)) {
> + perror("pwrite1");
> + exit(2);
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static inline void
> +complain(
> + loff_t offset,
> + char bad)
> +{
> + fprintf(stderr, "ASSERT: offset %llu should be 0x%x, got 0x%x!\n",
> + (unsigned long long)offset, GOOD_BYTE, bad);
> + abort();
> +}
> +
> +/* Make sure the destination file pagecache never changes. */
> +void *
> +mreader(
> + void *data)
> +{
> + volatile char *p;
> +
> + p = mmap(NULL, blksz, PROT_READ, MAP_SHARED, fd2, 0);
> + if (p == MAP_FAILED) {
> + perror("mmap");
> + exit(2);
> + }
> +
> + while (1) {
> + if (*(p + offset) != GOOD_BYTE)
> + complain(offset, *(p + offset));
> + }
> +}
> +
> +/* Make sure the destination file never changes. */
> +void *
> +preader(
> + void *data)
> +{
> + char v;
> + ssize_t sz;
> +
> + while (1) {
> + sz = pread(fd2, &v, sizeof(v), offset);
> + if (sz != sizeof(v)) {
> + perror("pwrite0");
> + exit(2);
> + }
> +
> + if (v != GOOD_BYTE)
> + complain(offset, v);
> + }
> +
> + return NULL;
> +}
> +
> +void
> +print_help(const char *progname)
> +{
> + printf("Usage: %s [-b blksz] [-c dir] [-n nr_ops] [-o offset] [-r] [-w] [-v]\n",
> + progname);
> + printf("-b sets the block size (default is autoconfigured)\n");
> + printf("-c chdir to this path before starting\n");
> + printf("-n controls the number of dedupe ops (default 10000)\n");
> + printf("-o reads and writes to this offset (default 37)\n");
> + printf("-r uses pread instead of mmap read.\n");
> + printf("-v prints status updates.\n");
> + printf("-w uses pwrite instead of mmap write.\n");
> +}
> +
> +int
> +main(
> + int argc,
> + char *argv[])
> +{
> + struct file_dedupe_range *fdr;
> + char *Xbuf;
> + void *(*reader_fn)(void *) = mreader;
> + void *(*writer_fn)(void *) = mwriter;
> + unsigned long same = 0;
> + unsigned long differs = 0;
> + unsigned long i, nr_ops = 10000;
> + ssize_t sz;
> + pthread_t reader, writer;
> + int verbose = 0;
> + int c;
> + int ret;
> +
> + while ((c = getopt(argc, argv, "b:c:n:o:rvw")) != -1) {
> + switch (c) {
> + case 'b':
> + errno = 0;
> + blksz = strtoul(optarg, NULL, 0);
> + if (errno) {
> + perror(optarg);
> + exit(1);
> + }
> + break;
> + case 'c':
> + ret = chdir(optarg);
> + if (ret) {
> + perror("chdir");
> + exit(1);
> + }
> + break;
> + case 'n':
> + errno = 0;
> + nr_ops = strtoul(optarg, NULL, 0);
> + if (errno) {
> + perror(optarg);
> + exit(1);
> + }
> + break;
> + case 'o':
> + errno = 0;
> + offset = strtoul(optarg, NULL, 0);
> + if (errno) {
> + perror(optarg);
> + exit(1);
> + }
> + break;
> + case 'r':
> + reader_fn = preader;
> + break;
> + case 'v':
> + verbose = 1;
> + break;
> + case 'w':
> + writer_fn = pwriter;
> + break;
> + default:
> + print_help(argv[0]);
> + exit(1);
> + break;
> + }
> + }
> +
> + fdr = malloc(sizeof(struct file_dedupe_range) +
> + sizeof(struct file_dedupe_range_info));
> + if (!fdr) {
> + perror("malloc");
> + exit(1);
> + }
> +
> + /* Initialize both files. */
> + fd1 = open("file1", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
> + if (fd1 < 0) {
> + perror("file1");
> + exit(1);
> + }
> +
> + fd2 = open("file2", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600);
> + if (fd2 < 0) {
> + perror("file2");
> + exit(1);
> + }
> +
> + if (blksz <= 0) {
> + struct stat statbuf;
> +
> + ret = fstat(fd1, &statbuf);
> + if (ret) {
> + perror("file1 stat");
> + exit(1);
> + }
> + blksz = statbuf.st_blksize;
> + }
> +
> + if (offset >= blksz) {
> + fprintf(stderr, "offset (%llu) < blksz (%llu)?\n",
> + (unsigned long long)offset,
> + (unsigned long long)blksz);
> + exit(1);
> + }
> +
> + Xbuf = malloc(blksz);
> + if (!Xbuf) {
> + perror("malloc buffer");
> + exit(1);
> + }
> + memset(Xbuf, GOOD_BYTE, blksz);
> +
> + sz = pwrite(fd1, Xbuf, blksz, 0);
> + if (sz != blksz) {
> + perror("file1 write");
> + exit(1);
> + }
> +
> + sz = pwrite(fd2, Xbuf, blksz, 0);
> + if (sz != blksz) {
> + perror("file2 write");
> + exit(1);
> + }
> +
> + ret = fsync(fd1);
> + if (ret) {
> + perror("file1 fsync");
> + exit(1);
> + }
> +
> + ret = fsync(fd2);
> + if (ret) {
> + perror("file2 fsync");
> + exit(1);
> + }
> +
> + /* Start our reader and writer threads. */
> + ret = pthread_create(&reader, NULL, reader_fn, NULL);
> + if (ret) {
> + fprintf(stderr, "rthread: %s\n", strerror(ret));
> + exit(1);
> + }
> +
> + ret = pthread_create(&writer, NULL, writer_fn, NULL);
> + if (ret) {
> + fprintf(stderr, "wthread: %s\n", strerror(ret));
> + exit(1);
> + }
> +
> + /*
> + * Now start deduping. If the contents match, fd1's blocks will be
> + * remapped into fd2, which is why the writer thread targets fd1 and
> + * the reader checks fd2 to make sure that none of fd1's writes ever
> + * make it into fd2.
> + */
> + for (i = 1; i <= nr_ops; i++) {
> + fdr->src_offset = 0;
> + fdr->src_length = blksz;
> + fdr->dest_count = 1;
> + fdr->reserved1 = 0;
> + fdr->reserved2 = 0;
> + fdr->info[0].dest_fd = fd2;
> + fdr->info[0].dest_offset = 0;
> + fdr->info[0].reserved = 0;
> +
> + ret = ioctl(fd1, FIDEDUPERANGE, fdr);
> + if (ret) {
> + perror("dedupe");
> + exit(2);
> + }
> +
> + switch (fdr->info[0].status) {
> + case FILE_DEDUPE_RANGE_DIFFERS:
> + differs++;
> + break;
> + case FILE_DEDUPE_RANGE_SAME:
> + same++;
> + break;
> + default:
> + fprintf(stderr, "deduperange: %s\n",
> + strerror(-fdr->info[0].status));
> + exit(2);
> + break;
> + }
> +
> + if (verbose && (i % 337) == 0)
> + printf("nr_ops: %lu; same: %lu; differs: %lu\n",
> + i, same, differs);
> + }
> +
> + if (verbose)
> + printf("nr_ops: %lu; same: %lu; differs: %lu\n", i - 1, same,
> + differs);
> +
> + /* Program termination will kill the threads and close the files. */
> + return 0;
> +}
> diff --git a/tests/generic/949 b/tests/generic/949
> new file mode 100755
> index 00000000..3951490b
> --- /dev/null
> +++ b/tests/generic/949
> @@ -0,0 +1,51 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 949
> +#
> +# Make sure that mmap and file writers racing with FIDEDUPERANGE cannot write
> +# to the file after the dedupe prep function has decided that the file contents
> +# are identical and we can therefore go ahead with the remapping.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/reflink
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch_dedupe
> +
> +rm -f $seqres.full
> +
> +nr_ops=$((TIME_FACTOR * 10000))
> +
> +# Format filesystem
> +_scratch_mkfs > $seqres.full
> +_scratch_mount
> +
> +# Test once with mmap writes
> +$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops
> +
> +# Test again with pwrites for the lulz
> +$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops -w
> +
> +echo Silence is golden.
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/949.out b/tests/generic/949.out
> new file mode 100644
> index 00000000..2998b46c
> --- /dev/null
> +++ b/tests/generic/949.out
> @@ -0,0 +1,2 @@
> +QA output created by 949
> +Silence is golden.
> diff --git a/tests/generic/group b/tests/generic/group
> index d5cfdd51..778aa8c4 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -630,3 +630,4 @@
> 625 auto quick verity
> 947 auto quick rw clone
> 948 auto quick rw copy_range
> +949 auto quick rw dedupe clone
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (7 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-14 18:06 ` Eryu Guan
2021-03-09 4:40 ` [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails Darrick J. Wong
2021-03-14 18:07 ` [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Eryu Guan
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: wenli xie, linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
wenli xie reported a buffer cache deadlock when an overlayfs is mounted
atop xfs and overlayfs tries to replace a single-nlink file with a
whiteout file. This test reproduces that deadlock.
Reported-by: wenli xie <wlxie7296@gmail.com>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/1300 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++
tests/generic/1300.out | 2 +
tests/generic/group | 1
3 files changed, 112 insertions(+)
create mode 100755 tests/generic/1300
create mode 100644 tests/generic/1300.out
diff --git a/tests/generic/1300 b/tests/generic/1300
new file mode 100755
index 00000000..10df44e3
--- /dev/null
+++ b/tests/generic/1300
@@ -0,0 +1,109 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 1300
+#
+# Reproducer for a deadlock in xfs_rename reported by Wenli Xie.
+#
+# When overlayfs is running on top of xfs and the user unlinks a file in the
+# overlay, overlayfs will create a whiteout inode and ask us to "rename" the
+# whiteout file atop the one being unlinked. If the file being unlinked loses
+# its one nlink, we then have to put the inode on the unlinked list.
+#
+# This requires us to grab the AGI buffer of the whiteout inode to take it
+# off the unlinked list (which is where whiteouts are created) and to grab
+# the AGI buffer of the file being deleted. If the whiteout was created in
+# a higher numbered AG than the file being deleted, we'll lock the AGIs in
+# the wrong order and deadlock.
+#
+# Note that this test doesn't do anything xfs-specific so it's a generic test.
+# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock
+# in xfs_rename").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ stop_workers
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs."
+
+modprobe -q overlay
+grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs."
+
+rm -f $seqres.full
+
+_scratch_mkfs >> $seqres.full
+_scratch_mount
+
+mkdir $SCRATCH_MNT/lowerdir
+mkdir $SCRATCH_MNT/lowerdir1
+mkdir $SCRATCH_MNT/lowerdir/etc
+mkdir $SCRATCH_MNT/workers
+echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf
+touch $SCRATCH_MNT/running
+
+stop_workers() {
+ test -e $SCRATCH_MNT/running || return
+ rm -f $SCRATCH_MNT/running
+
+ while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do
+ wait
+ done
+}
+
+worker() {
+ local tag="$1"
+ local mergedir="$SCRATCH_MNT/merged$tag"
+ local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1"
+ local u="upperdir=$SCRATCH_MNT/upperdir$tag"
+ local w="workdir=$SCRATCH_MNT/workdir$tag"
+ local i="index=off,nfs_export=off"
+
+ touch $SCRATCH_MNT/workers/$tag
+ while test -e $SCRATCH_MNT/running; do
+ rm -rf $SCRATCH_MNT/merged$tag
+ rm -rf $SCRATCH_MNT/upperdir$tag
+ rm -rf $SCRATCH_MNT/workdir$tag
+ mkdir $SCRATCH_MNT/merged$tag
+ mkdir $SCRATCH_MNT/workdir$tag
+ mkdir $SCRATCH_MNT/upperdir$tag
+
+ mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir
+ mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
+ touch $mergedir/etc/access.conf
+ mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
+ touch $mergedir/etc/access.conf
+ umount $mergedir
+ done
+ rm -f $SCRATCH_MNT/workers/$tag
+}
+
+for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do
+ worker $i &
+done
+
+sleep $((30 * TIME_FACTOR))
+stop_workers
+
+echo Silence is golden.
+# success, all done
+status=0
+exit
diff --git a/tests/generic/1300.out b/tests/generic/1300.out
new file mode 100644
index 00000000..5805d30d
--- /dev/null
+++ b/tests/generic/1300.out
@@ -0,0 +1,2 @@
+QA output created by 1300
+Silence is golden.
diff --git a/tests/generic/group b/tests/generic/group
index 778aa8c4..2233a59d 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -631,3 +631,4 @@
947 auto quick rw clone
948 auto quick rw copy_range
949 auto quick rw dedupe clone
+1300 auto rw overlay
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files
2021-03-09 4:40 ` [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files Darrick J. Wong
@ 2021-03-14 18:06 ` Eryu Guan
2021-03-15 16:54 ` Darrick J. Wong
0 siblings, 1 reply; 27+ messages in thread
From: Eryu Guan @ 2021-03-14 18:06 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, wenli xie, linux-xfs, fstests
On Mon, Mar 08, 2021 at 08:40:53PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> wenli xie reported a buffer cache deadlock when an overlayfs is mounted
> atop xfs and overlayfs tries to replace a single-nlink file with a
> whiteout file. This test reproduces that deadlock.
>
> Reported-by: wenli xie <wlxie7296@gmail.com>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> tests/generic/1300 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++
> tests/generic/1300.out | 2 +
> tests/generic/group | 1
> 3 files changed, 112 insertions(+)
> create mode 100755 tests/generic/1300
> create mode 100644 tests/generic/1300.out
>
>
> diff --git a/tests/generic/1300 b/tests/generic/1300
> new file mode 100755
> index 00000000..10df44e3
> --- /dev/null
> +++ b/tests/generic/1300
> @@ -0,0 +1,109 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 1300
> +#
> +# Reproducer for a deadlock in xfs_rename reported by Wenli Xie.
> +#
> +# When overlayfs is running on top of xfs and the user unlinks a file in the
> +# overlay, overlayfs will create a whiteout inode and ask us to "rename" the
> +# whiteout file atop the one being unlinked. If the file being unlinked loses
> +# its one nlink, we then have to put the inode on the unlinked list.
> +#
> +# This requires us to grab the AGI buffer of the whiteout inode to take it
> +# off the unlinked list (which is where whiteouts are created) and to grab
> +# the AGI buffer of the file being deleted. If the whiteout was created in
> +# a higher numbered AG than the file being deleted, we'll lock the AGIs in
> +# the wrong order and deadlock.
> +#
> +# Note that this test doesn't do anything xfs-specific so it's a generic test.
> +# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock
> +# in xfs_rename").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + stop_workers
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch
> +test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs."
> +
> +modprobe -q overlay
> +grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs."
We have _require_ext2() and _require_tmpfs(), I think it's time to
refactor them into a new _require_filesystem() helper?
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >> $seqres.full
> +_scratch_mount
Also, doing overlay mount requires the underlying to export d_type,
perhaps we should do '_supports_filetype $SCRATCH_MNT' here, otherwise
filesystems like nfs would fail the test.
> +
> +mkdir $SCRATCH_MNT/lowerdir
> +mkdir $SCRATCH_MNT/lowerdir1
> +mkdir $SCRATCH_MNT/lowerdir/etc
> +mkdir $SCRATCH_MNT/workers
> +echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf
> +touch $SCRATCH_MNT/running
> +
> +stop_workers() {
> + test -e $SCRATCH_MNT/running || return
> + rm -f $SCRATCH_MNT/running
> +
> + while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do
> + wait
> + done
> +}
> +
> +worker() {
> + local tag="$1"
> + local mergedir="$SCRATCH_MNT/merged$tag"
> + local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1"
> + local u="upperdir=$SCRATCH_MNT/upperdir$tag"
> + local w="workdir=$SCRATCH_MNT/workdir$tag"
> + local i="index=off,nfs_export=off"
> +
> + touch $SCRATCH_MNT/workers/$tag
> + while test -e $SCRATCH_MNT/running; do
> + rm -rf $SCRATCH_MNT/merged$tag
> + rm -rf $SCRATCH_MNT/upperdir$tag
> + rm -rf $SCRATCH_MNT/workdir$tag
> + mkdir $SCRATCH_MNT/merged$tag
> + mkdir $SCRATCH_MNT/workdir$tag
> + mkdir $SCRATCH_MNT/upperdir$tag
> +
> + mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir
> + mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> + touch $mergedir/etc/access.conf
> + mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> + touch $mergedir/etc/access.conf
> + umount $mergedir
> + done
> + rm -f $SCRATCH_MNT/workers/$tag
> +}
> +
> +for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do
> + worker $i &
> +done
> +
> +sleep $((30 * TIME_FACTOR))
> +stop_workers
> +
> +echo Silence is golden.
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/1300.out b/tests/generic/1300.out
> new file mode 100644
> index 00000000..5805d30d
> --- /dev/null
> +++ b/tests/generic/1300.out
> @@ -0,0 +1,2 @@
> +QA output created by 1300
> +Silence is golden.
> diff --git a/tests/generic/group b/tests/generic/group
> index 778aa8c4..2233a59d 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -631,3 +631,4 @@
> 947 auto quick rw clone
> 948 auto quick rw copy_range
> 949 auto quick rw dedupe clone
> +1300 auto rw overlay
Also add 'rename' group?
Thanks,
Eryu
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files
2021-03-14 18:06 ` Eryu Guan
@ 2021-03-15 16:54 ` Darrick J. Wong
0 siblings, 0 replies; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-15 16:54 UTC (permalink / raw)
To: Eryu Guan; +Cc: guaneryu, wenli xie, linux-xfs, fstests
On Mon, Mar 15, 2021 at 02:06:35AM +0800, Eryu Guan wrote:
> On Mon, Mar 08, 2021 at 08:40:53PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > wenli xie reported a buffer cache deadlock when an overlayfs is mounted
> > atop xfs and overlayfs tries to replace a single-nlink file with a
> > whiteout file. This test reproduces that deadlock.
> >
> > Reported-by: wenli xie <wlxie7296@gmail.com>
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> > tests/generic/1300 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/generic/1300.out | 2 +
> > tests/generic/group | 1
> > 3 files changed, 112 insertions(+)
> > create mode 100755 tests/generic/1300
> > create mode 100644 tests/generic/1300.out
> >
> >
> > diff --git a/tests/generic/1300 b/tests/generic/1300
> > new file mode 100755
> > index 00000000..10df44e3
> > --- /dev/null
> > +++ b/tests/generic/1300
> > @@ -0,0 +1,109 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle. All Rights Reserved.
> > +#
> > +# FS QA Test No. 1300
> > +#
> > +# Reproducer for a deadlock in xfs_rename reported by Wenli Xie.
> > +#
> > +# When overlayfs is running on top of xfs and the user unlinks a file in the
> > +# overlay, overlayfs will create a whiteout inode and ask us to "rename" the
> > +# whiteout file atop the one being unlinked. If the file being unlinked loses
> > +# its one nlink, we then have to put the inode on the unlinked list.
> > +#
> > +# This requires us to grab the AGI buffer of the whiteout inode to take it
> > +# off the unlinked list (which is where whiteouts are created) and to grab
> > +# the AGI buffer of the file being deleted. If the whiteout was created in
> > +# a higher numbered AG than the file being deleted, we'll lock the AGIs in
> > +# the wrong order and deadlock.
> > +#
> > +# Note that this test doesn't do anything xfs-specific so it's a generic test.
> > +# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock
> > +# in xfs_rename").
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + stop_workers
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_scratch
> > +test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs."
> > +
> > +modprobe -q overlay
> > +grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs."
>
> We have _require_ext2() and _require_tmpfs(), I think it's time to
> refactor them into a new _require_filesystem() helper?
Yeah, I'll refactor them into a new _require_extra_fs helper.
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs >> $seqres.full
> > +_scratch_mount
>
> Also, doing overlay mount requires the underlying to export d_type,
> perhaps we should do '_supports_filetype $SCRATCH_MNT' here, otherwise
> filesystems like nfs would fail the test.
Oh! Thank you for pointing that out.
>
> > +
> > +mkdir $SCRATCH_MNT/lowerdir
> > +mkdir $SCRATCH_MNT/lowerdir1
> > +mkdir $SCRATCH_MNT/lowerdir/etc
> > +mkdir $SCRATCH_MNT/workers
> > +echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf
> > +touch $SCRATCH_MNT/running
> > +
> > +stop_workers() {
> > + test -e $SCRATCH_MNT/running || return
> > + rm -f $SCRATCH_MNT/running
> > +
> > + while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do
> > + wait
> > + done
> > +}
> > +
> > +worker() {
> > + local tag="$1"
> > + local mergedir="$SCRATCH_MNT/merged$tag"
> > + local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1"
> > + local u="upperdir=$SCRATCH_MNT/upperdir$tag"
> > + local w="workdir=$SCRATCH_MNT/workdir$tag"
> > + local i="index=off,nfs_export=off"
> > +
> > + touch $SCRATCH_MNT/workers/$tag
> > + while test -e $SCRATCH_MNT/running; do
> > + rm -rf $SCRATCH_MNT/merged$tag
> > + rm -rf $SCRATCH_MNT/upperdir$tag
> > + rm -rf $SCRATCH_MNT/workdir$tag
> > + mkdir $SCRATCH_MNT/merged$tag
> > + mkdir $SCRATCH_MNT/workdir$tag
> > + mkdir $SCRATCH_MNT/upperdir$tag
> > +
> > + mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir
> > + mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> > + touch $mergedir/etc/access.conf
> > + mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak
> > + touch $mergedir/etc/access.conf
> > + umount $mergedir
> > + done
> > + rm -f $SCRATCH_MNT/workers/$tag
> > +}
> > +
> > +for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do
> > + worker $i &
> > +done
> > +
> > +sleep $((30 * TIME_FACTOR))
> > +stop_workers
> > +
> > +echo Silence is golden.
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/1300.out b/tests/generic/1300.out
> > new file mode 100644
> > index 00000000..5805d30d
> > --- /dev/null
> > +++ b/tests/generic/1300.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 1300
> > +Silence is golden.
> > diff --git a/tests/generic/group b/tests/generic/group
> > index 778aa8c4..2233a59d 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -631,3 +631,4 @@
> > 947 auto quick rw clone
> > 948 auto quick rw copy_range
> > 949 auto quick rw dedupe clone
> > +1300 auto rw overlay
>
> Also add 'rename' group?
Done.
--D
> Thanks,
> Eryu
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (8 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files Darrick J. Wong
@ 2021-03-09 4:40 ` Darrick J. Wong
2021-03-12 8:49 ` Chandan Babu R
2021-03-14 18:07 ` [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Eryu Guan
10 siblings, 1 reply; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-09 4:40 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
This is a regression test for a bug in the XFS implementation of
FSSETXATTR. When we try to change a file's project id, the quota
reservation code will update the incore quota reservations for delayed
allocation blocks. Unfortunately, it does this before we finish
validating all the FSSETXATTR parameters, which means that if we decide
to bail out, we also fail to undo the incore changes.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
.gitignore | 1 +
src/Makefile | 2 +
src/chprojid_fail.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
tests/xfs/765 | 71 +++++++++++++++++++++++++++++++++++++++
tests/xfs/765.out | 4 ++
tests/xfs/group | 1 +
6 files changed, 170 insertions(+), 1 deletion(-)
create mode 100644 src/chprojid_fail.c
create mode 100755 tests/xfs/765
create mode 100644 tests/xfs/765.out
diff --git a/.gitignore b/.gitignore
index 03c03be5..3af8e207 100644
--- a/.gitignore
+++ b/.gitignore
@@ -58,6 +58,7 @@
/src/bulkstat_null_ocount
/src/bulkstat_unlink_test
/src/bulkstat_unlink_test_modified
+/src/chprojid_fail
/src/cloner
/src/dbtest
/src/devzero
diff --git a/src/Makefile b/src/Makefile
index 38ee6718..3d729a34 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -29,7 +29,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
dio-invalidate-cache stat_test t_encrypted_d_revalidate \
attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
- fscrypt-crypt-util bulkstat_null_ocount splice-test
+ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail
SUBDIRS = log-writes perf
diff --git a/src/chprojid_fail.c b/src/chprojid_fail.c
new file mode 100644
index 00000000..8c5b5fee
--- /dev/null
+++ b/src/chprojid_fail.c
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2021 Oracle. All Rights Reserved.
+ * Author: Darrick J. Wong <djwong@kernel.org>
+ *
+ * Regression test for failing to undo delalloc quota reservations when
+ * changing project id and we fail some other FSSETXATTR validation.
+ */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <linux/fs.h>
+
+static char zerobuf[65536];
+
+int
+main(
+ int argc,
+ char *argv[])
+{
+ struct fsxattr fa;
+ ssize_t sz;
+ int fd, ret;
+
+ if (argc < 2) {
+ printf("Usage: %s filename\n", argv[0]);
+ return 1;
+ }
+
+ fd = open(argv[1], O_CREAT | O_TRUNC | O_RDWR, 0600);
+ if (fd < 0) {
+ perror(argv[1]);
+ return 2;
+ }
+
+ /* Zero the project id and the extent size hint. */
+ ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
+ if (ret) {
+ perror("FSGETXATTR check file");
+ return 2;
+ }
+
+ if (fa.fsx_projid != 0 || fa.fsx_extsize != 0) {
+ fa.fsx_projid = 0;
+ fa.fsx_extsize = 0;
+ ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
+ if (ret) {
+ perror("FSSETXATTR zeroing");
+ return 2;
+ }
+ }
+
+ /* Dirty a few kb of a file to create delalloc extents. */
+ sz = write(fd, zerobuf, sizeof(zerobuf));
+ if (sz != sizeof(zerobuf)) {
+ perror("delalloc write");
+ return 2;
+ }
+
+ /*
+ * The regression we're trying to test happens when the fsxattr input
+ * validation decides to bail out after the chown quota reservation has
+ * been made on a file containing delalloc extents. Extent size hints
+ * can't be set on non-empty files and we can't check the value until
+ * we've reserved resources and taken the file's ILOCK, so this is a
+ * perfect vector for triggering this condition. In this way we set up
+ * a FSSETXATTR call that will fail.
+ */
+ ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
+ if (ret) {
+ perror("FSGETXATTR");
+ return 2;
+ }
+
+ fa.fsx_projid = 23652;
+ fa.fsx_extsize = 2;
+ fa.fsx_xflags |= FS_XFLAG_EXTSIZE;
+
+ ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
+ if (ret) {
+ printf("FSSETXATTRR should fail: %s\n", strerror(errno));
+ return 0;
+ }
+
+ /* Uhoh, that FSSETXATTR call should have failed! */
+ return 3;
+}
diff --git a/tests/xfs/765 b/tests/xfs/765
new file mode 100755
index 00000000..68b89ce3
--- /dev/null
+++ b/tests/xfs/765
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 765
+#
+# Regression test for failing to undo delalloc quota reservations when changing
+# project id but we fail some other part of FSSETXATTR validation. If we fail
+# the test, we trip debugging assertions in dmesg. This is a regression test
+# for commit 1aecf3734a95 ("xfs: fix chown leaking delalloc quota blocks when
+# fssetxattr fails").
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/quota
+
+# real QA test starts here
+_supported_fs xfs
+_require_command "$FILEFRAG_PROG" filefrag
+_require_test_program "chprojid_fail"
+_require_quota
+_require_scratch
+
+rm -f $seqres.full
+
+echo "Format filesystem" | tee -a $seqres.full
+_scratch_mkfs > $seqres.full
+_qmount_option 'prjquota'
+_qmount
+_require_prjquota $SCRATCH_DEV
+
+# Make sure that a regular buffered write produces delalloc reservations.
+$XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null
+$FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \
+ _notrun "test requires delayed allocation writes"
+rm -f $SCRATCH_MNT/testy
+
+echo "Run test program"
+$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
+$here/src/chprojid_fail $SCRATCH_MNT/blah
+
+# The regression we're testing for is an accounting bug involving delalloc
+# reservations. FSSETXATTR does not itself cause dirty data writeback, so we
+# assume that if the file still has delalloc extents, then it must have had
+# them when chprojid_fail was running, and therefore the test was set up
+# correctly. There's a slight chance that background writeback can sneak in
+# and flush the file, but this should be a small enough gap.
+$FILEFRAG_PROG -v $SCRATCH_MNT/blah 2>&1 | grep -q delalloc || \
+ echo "file didn't get delalloc extents, test invalid?"
+
+# Make a note of current quota status for diagnostic purposes
+$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/765.out b/tests/xfs/765.out
new file mode 100644
index 00000000..d5f8fc11
--- /dev/null
+++ b/tests/xfs/765.out
@@ -0,0 +1,4 @@
+QA output created by 765
+Format filesystem
+Run test program
+FSSETXATTRR should fail: Invalid argument
diff --git a/tests/xfs/group b/tests/xfs/group
index d7aafc94..84468d10 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -505,4 +505,5 @@
760 auto quick rw collapse punch insert zero prealloc
761 auto quick realtime
763 auto quick rw realtime
+765 auto quick quota
915 auto quick quota
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails
2021-03-09 4:40 ` [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails Darrick J. Wong
@ 2021-03-12 8:49 ` Chandan Babu R
0 siblings, 0 replies; 27+ messages in thread
From: Chandan Babu R @ 2021-03-12 8:49 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 09 Mar 2021 at 10:10, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> This is a regression test for a bug in the XFS implementation of
> FSSETXATTR. When we try to change a file's project id, the quota
> reservation code will update the incore quota reservations for delayed
> allocation blocks. Unfortunately, it does this before we finish
> validating all the FSSETXATTR parameters, which means that if we decide
> to bail out, we also fail to undo the incore changes.
>
Looks good to me.
Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> .gitignore | 1 +
> src/Makefile | 2 +
> src/chprojid_fail.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/765 | 71 +++++++++++++++++++++++++++++++++++++++
> tests/xfs/765.out | 4 ++
> tests/xfs/group | 1 +
> 6 files changed, 170 insertions(+), 1 deletion(-)
> create mode 100644 src/chprojid_fail.c
> create mode 100755 tests/xfs/765
> create mode 100644 tests/xfs/765.out
>
>
> diff --git a/.gitignore b/.gitignore
> index 03c03be5..3af8e207 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -58,6 +58,7 @@
> /src/bulkstat_null_ocount
> /src/bulkstat_unlink_test
> /src/bulkstat_unlink_test_modified
> +/src/chprojid_fail
> /src/cloner
> /src/dbtest
> /src/devzero
> diff --git a/src/Makefile b/src/Makefile
> index 38ee6718..3d729a34 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -29,7 +29,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
> attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \
> dio-invalidate-cache stat_test t_encrypted_d_revalidate \
> attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \
> - fscrypt-crypt-util bulkstat_null_ocount splice-test
> + fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail
>
> SUBDIRS = log-writes perf
>
> diff --git a/src/chprojid_fail.c b/src/chprojid_fail.c
> new file mode 100644
> index 00000000..8c5b5fee
> --- /dev/null
> +++ b/src/chprojid_fail.c
> @@ -0,0 +1,92 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (c) 2021 Oracle. All Rights Reserved.
> + * Author: Darrick J. Wong <djwong@kernel.org>
> + *
> + * Regression test for failing to undo delalloc quota reservations when
> + * changing project id and we fail some other FSSETXATTR validation.
> + */
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <unistd.h>
> +#include <sys/ioctl.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <errno.h>
> +#include <linux/fs.h>
> +
> +static char zerobuf[65536];
> +
> +int
> +main(
> + int argc,
> + char *argv[])
> +{
> + struct fsxattr fa;
> + ssize_t sz;
> + int fd, ret;
> +
> + if (argc < 2) {
> + printf("Usage: %s filename\n", argv[0]);
> + return 1;
> + }
> +
> + fd = open(argv[1], O_CREAT | O_TRUNC | O_RDWR, 0600);
> + if (fd < 0) {
> + perror(argv[1]);
> + return 2;
> + }
> +
> + /* Zero the project id and the extent size hint. */
> + ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
> + if (ret) {
> + perror("FSGETXATTR check file");
> + return 2;
> + }
> +
> + if (fa.fsx_projid != 0 || fa.fsx_extsize != 0) {
> + fa.fsx_projid = 0;
> + fa.fsx_extsize = 0;
> + ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
> + if (ret) {
> + perror("FSSETXATTR zeroing");
> + return 2;
> + }
> + }
> +
> + /* Dirty a few kb of a file to create delalloc extents. */
> + sz = write(fd, zerobuf, sizeof(zerobuf));
> + if (sz != sizeof(zerobuf)) {
> + perror("delalloc write");
> + return 2;
> + }
> +
> + /*
> + * The regression we're trying to test happens when the fsxattr input
> + * validation decides to bail out after the chown quota reservation has
> + * been made on a file containing delalloc extents. Extent size hints
> + * can't be set on non-empty files and we can't check the value until
> + * we've reserved resources and taken the file's ILOCK, so this is a
> + * perfect vector for triggering this condition. In this way we set up
> + * a FSSETXATTR call that will fail.
> + */
> + ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa);
> + if (ret) {
> + perror("FSGETXATTR");
> + return 2;
> + }
> +
> + fa.fsx_projid = 23652;
> + fa.fsx_extsize = 2;
> + fa.fsx_xflags |= FS_XFLAG_EXTSIZE;
> +
> + ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa);
> + if (ret) {
> + printf("FSSETXATTRR should fail: %s\n", strerror(errno));
> + return 0;
> + }
> +
> + /* Uhoh, that FSSETXATTR call should have failed! */
> + return 3;
> +}
> diff --git a/tests/xfs/765 b/tests/xfs/765
> new file mode 100755
> index 00000000..68b89ce3
> --- /dev/null
> +++ b/tests/xfs/765
> @@ -0,0 +1,71 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle. All Rights Reserved.
> +#
> +# FS QA Test No. 765
> +#
> +# Regression test for failing to undo delalloc quota reservations when changing
> +# project id but we fail some other part of FSSETXATTR validation. If we fail
> +# the test, we trip debugging assertions in dmesg. This is a regression test
> +# for commit 1aecf3734a95 ("xfs: fix chown leaking delalloc quota blocks when
> +# fssetxattr fails").
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/quota
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_command "$FILEFRAG_PROG" filefrag
> +_require_test_program "chprojid_fail"
> +_require_quota
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +echo "Format filesystem" | tee -a $seqres.full
> +_scratch_mkfs > $seqres.full
> +_qmount_option 'prjquota'
> +_qmount
> +_require_prjquota $SCRATCH_DEV
> +
> +# Make sure that a regular buffered write produces delalloc reservations.
> +$XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null
> +$FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \
> + _notrun "test requires delayed allocation writes"
> +rm -f $SCRATCH_MNT/testy
> +
> +echo "Run test program"
> +$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
> +$here/src/chprojid_fail $SCRATCH_MNT/blah
> +
> +# The regression we're testing for is an accounting bug involving delalloc
> +# reservations. FSSETXATTR does not itself cause dirty data writeback, so we
> +# assume that if the file still has delalloc extents, then it must have had
> +# them when chprojid_fail was running, and therefore the test was set up
> +# correctly. There's a slight chance that background writeback can sneak in
> +# and flush the file, but this should be a small enough gap.
> +$FILEFRAG_PROG -v $SCRATCH_MNT/blah 2>&1 | grep -q delalloc || \
> + echo "file didn't get delalloc extents, test invalid?"
> +
> +# Make a note of current quota status for diagnostic purposes
> +$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/765.out b/tests/xfs/765.out
> new file mode 100644
> index 00000000..d5f8fc11
> --- /dev/null
> +++ b/tests/xfs/765.out
> @@ -0,0 +1,4 @@
> +QA output created by 765
> +Format filesystem
> +Run test program
> +FSSETXATTRR should fail: Invalid argument
> diff --git a/tests/xfs/group b/tests/xfs/group
> index d7aafc94..84468d10 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -505,4 +505,5 @@
> 760 auto quick rw collapse punch insert zero prealloc
> 761 auto quick realtime
> 763 auto quick rw realtime
> +765 auto quick quota
> 915 auto quick quota
--
chandan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12
2021-03-09 4:40 [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Darrick J. Wong
` (9 preceding siblings ...)
2021-03-09 4:40 ` [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails Darrick J. Wong
@ 2021-03-14 18:07 ` Eryu Guan
2021-03-14 21:36 ` Darrick J. Wong
10 siblings, 1 reply; 27+ messages in thread
From: Eryu Guan @ 2021-03-14 18:07 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, wenli xie, Chandan Babu R, linux-xfs, fstests
On Mon, Mar 08, 2021 at 08:40:03PM -0800, Darrick J. Wong wrote:
> Hi all,
>
> Here are new tests for problems that were fixed in upstream Linux
> between 5.9 and 5.12.
I've applied all patches except 3/10 and 9/10, thanks for the tests and
fixes!
Eryu
>
> If you're going to start using this mess, you probably ought to just
> pull from my git trees, which are linked below.
>
> This is an extraordinary way to destroy everything. Enjoy!
> Comments and questions are, as always, welcome.
>
> --D
>
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=kernel-regressions
> ---
> .gitignore | 1
> common/filter | 24 +++
> src/Makefile | 4 -
> src/chprojid_fail.c | 92 ++++++++++++
> src/deduperace.c | 370 ++++++++++++++++++++++++++++++++++++++++++++++++
> tests/generic/1300 | 109 ++++++++++++++
> tests/generic/1300.out | 2
> tests/generic/947 | 118 +++++++++++++++
> tests/generic/947.out | 15 ++
> tests/generic/948 | 92 ++++++++++++
> tests/generic/948.out | 9 +
> tests/generic/949 | 51 +++++++
> tests/generic/949.out | 2
> tests/generic/group | 4 +
> tests/xfs/050 | 30 +---
> tests/xfs/122 | 1
> tests/xfs/122.out | 1
> tests/xfs/299 | 30 +---
> tests/xfs/758 | 59 ++++++++
> tests/xfs/758.out | 2
> tests/xfs/759 | 100 +++++++++++++
> tests/xfs/759.out | 2
> tests/xfs/760 | 68 +++++++++
> tests/xfs/760.out | 9 +
> tests/xfs/761 | 45 ++++++
> tests/xfs/761.out | 1
> tests/xfs/765 | 71 +++++++++
> tests/xfs/765.out | 4 +
> tests/xfs/915 | 162 +++++++++++++++++++++
> tests/xfs/915.out | 151 ++++++++++++++++++++
> tests/xfs/group | 6 +
> 31 files changed, 1584 insertions(+), 51 deletions(-)
> create mode 100644 src/chprojid_fail.c
> create mode 100644 src/deduperace.c
> create mode 100755 tests/generic/1300
> create mode 100644 tests/generic/1300.out
> create mode 100755 tests/generic/947
> create mode 100644 tests/generic/947.out
> create mode 100755 tests/generic/948
> create mode 100644 tests/generic/948.out
> create mode 100755 tests/generic/949
> create mode 100644 tests/generic/949.out
> create mode 100755 tests/xfs/758
> create mode 100644 tests/xfs/758.out
> create mode 100755 tests/xfs/759
> create mode 100644 tests/xfs/759.out
> create mode 100755 tests/xfs/760
> create mode 100644 tests/xfs/760.out
> create mode 100755 tests/xfs/761
> create mode 100644 tests/xfs/761.out
> create mode 100755 tests/xfs/765
> create mode 100644 tests/xfs/765.out
> create mode 100755 tests/xfs/915
> create mode 100644 tests/xfs/915.out
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12
2021-03-14 18:07 ` [PATCHSET 00/10] fstests: test kernel regressions fixed in 5.12 Eryu Guan
@ 2021-03-14 21:36 ` Darrick J. Wong
0 siblings, 0 replies; 27+ messages in thread
From: Darrick J. Wong @ 2021-03-14 21:36 UTC (permalink / raw)
To: Eryu Guan; +Cc: guaneryu, wenli xie, Chandan Babu R, linux-xfs, fstests
On Mon, Mar 15, 2021 at 02:07:44AM +0800, Eryu Guan wrote:
> On Mon, Mar 08, 2021 at 08:40:03PM -0800, Darrick J. Wong wrote:
> > Hi all,
> >
> > Here are new tests for problems that were fixed in upstream Linux
> > between 5.9 and 5.12.
>
> I've applied all patches except 3/10 and 9/10, thanks for the tests and
> fixes!
Thank /you/ for putting them in! I'll revise those two and have a new
submission in a few days.
--D
> Eryu
>
> >
> > If you're going to start using this mess, you probably ought to just
> > pull from my git trees, which are linked below.
> >
> > This is an extraordinary way to destroy everything. Enjoy!
> > Comments and questions are, as always, welcome.
> >
> > --D
> >
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=kernel-regressions
> > ---
> > .gitignore | 1
> > common/filter | 24 +++
> > src/Makefile | 4 -
> > src/chprojid_fail.c | 92 ++++++++++++
> > src/deduperace.c | 370 ++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/generic/1300 | 109 ++++++++++++++
> > tests/generic/1300.out | 2
> > tests/generic/947 | 118 +++++++++++++++
> > tests/generic/947.out | 15 ++
> > tests/generic/948 | 92 ++++++++++++
> > tests/generic/948.out | 9 +
> > tests/generic/949 | 51 +++++++
> > tests/generic/949.out | 2
> > tests/generic/group | 4 +
> > tests/xfs/050 | 30 +---
> > tests/xfs/122 | 1
> > tests/xfs/122.out | 1
> > tests/xfs/299 | 30 +---
> > tests/xfs/758 | 59 ++++++++
> > tests/xfs/758.out | 2
> > tests/xfs/759 | 100 +++++++++++++
> > tests/xfs/759.out | 2
> > tests/xfs/760 | 68 +++++++++
> > tests/xfs/760.out | 9 +
> > tests/xfs/761 | 45 ++++++
> > tests/xfs/761.out | 1
> > tests/xfs/765 | 71 +++++++++
> > tests/xfs/765.out | 4 +
> > tests/xfs/915 | 162 +++++++++++++++++++++
> > tests/xfs/915.out | 151 ++++++++++++++++++++
> > tests/xfs/group | 6 +
> > 31 files changed, 1584 insertions(+), 51 deletions(-)
> > create mode 100644 src/chprojid_fail.c
> > create mode 100644 src/deduperace.c
> > create mode 100755 tests/generic/1300
> > create mode 100644 tests/generic/1300.out
> > create mode 100755 tests/generic/947
> > create mode 100644 tests/generic/947.out
> > create mode 100755 tests/generic/948
> > create mode 100644 tests/generic/948.out
> > create mode 100755 tests/generic/949
> > create mode 100644 tests/generic/949.out
> > create mode 100755 tests/xfs/758
> > create mode 100644 tests/xfs/758.out
> > create mode 100755 tests/xfs/759
> > create mode 100644 tests/xfs/759.out
> > create mode 100755 tests/xfs/760
> > create mode 100644 tests/xfs/760.out
> > create mode 100755 tests/xfs/761
> > create mode 100644 tests/xfs/761.out
> > create mode 100755 tests/xfs/765
> > create mode 100644 tests/xfs/765.out
> > create mode 100755 tests/xfs/915
> > create mode 100644 tests/xfs/915.out
^ permalink raw reply [flat|nested] 27+ messages in thread