* [PATCH 0/5] xfstests: fixes and reflink quota tests
@ 2018-01-24 23:53 Darrick J. Wong
2018-01-24 23:53 ` [PATCH 1/5] common/rc: report kmemleak errors Darrick J. Wong
` (4 more replies)
0 siblings, 5 replies; 11+ messages in thread
From: Darrick J. Wong @ 2018-01-24 23:53 UTC (permalink / raw)
To: eguan, darrick.wong; +Cc: linux-xfs, fstests
Hi all,
This series contains the kmemleak checks for xfstests, a bunch of minor
fixes to existing tests, and some new tests that check quota accounting
regressions in xfs. The quota accounting tests will fail until the
series "xfs: reflink/scrub/quota fixes" goes into 4.16.
--D
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH 1/5] common/rc: report kmemleak errors 2018-01-24 23:53 [PATCH 0/5] xfstests: fixes and reflink quota tests Darrick J. Wong @ 2018-01-24 23:53 ` Darrick J. Wong 2018-01-24 23:53 ` [PATCH 2/5] xfs/122: fix xfs header ordering problems Darrick J. Wong ` (3 subsequent siblings) 4 siblings, 0 replies; 11+ messages in thread From: Darrick J. Wong @ 2018-01-24 23:53 UTC (permalink / raw) To: eguan, darrick.wong; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> If kmemleak is enabled, scan and report memory leaks after every test. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- check | 2 ++ common/rc | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+) diff --git a/check b/check index 9868d22..b7b3d57 100755 --- a/check +++ b/check @@ -499,6 +499,7 @@ _check_filesystems() fi } +_init_kmemleak _prepare_test_list if $OPTIONS_HAVE_SECTIONS; then @@ -795,6 +796,7 @@ for section in $HOST_OPTIONS_SECTIONS; do n_try=`expr $n_try + 1` _check_filesystems _check_dmesg || err=true + _check_kmemleak || err=true fi fi diff --git a/common/rc b/common/rc index 880555c..0d0def7 100644 --- a/common/rc +++ b/common/rc @@ -3452,6 +3452,66 @@ _check_dmesg() fi } +# capture the kmemleak report +_capture_kmemleak() +{ + local _kern_knob="${DEBUGFS_MNT}/kmemleak" + local _leak_file="$1" + + # Tell the kernel to scan for memory leaks. Apparently the write + # returns before the scan is complete, so do it twice in the hopes + # that twice is enough to capture all the leaks. + echo "scan" > "${_kern_knob}" + cat "${_kern_knob}" > /dev/null + echo "scan" > "${_kern_knob}" + cat "${_kern_knob}" > "${_leak_file}.tmp" + if [ -s "${_leak_file}.tmp" ]; then + cat > "${_leak_file}" << ENDL +EXPERIMENTAL kmemleak reported some memory leaks! Due to the way kmemleak +works, the leak might be from an earlier test, or something totally unrelated. +ENDL + cat "${_leak_file}.tmp" >> "${_leak_file}" + rm -rf "${_leak_file}.tmp" + fi + echo "clear" > "${_kern_knob}" +} + +# set up kmemleak +_init_kmemleak() +{ + local _kern_knob="${DEBUGFS_MNT}/kmemleak" + + if [ ! -w "${_kern_knob}" ]; then + return 0 + fi + + # Disable the automatic scan so that we can control it completely, + # then dump all the leaks recorded so far. + echo "scan=off" > "${_kern_knob}" + _capture_kmemleak /dev/null +} + +# check kmemleak log +_check_kmemleak() +{ + local _kern_knob="${DEBUGFS_MNT}/kmemleak" + local _leak_file="${seqres}.kmemleak" + + if [ ! -w "${_kern_knob}" ]; then + return 0 + fi + + # Capture and report any leaks + _capture_kmemleak "${_leak_file}" + if [ -s "${_leak_file}" ]; then + _dump_err "_check_kmemleak: something found in kmemleak (see ${_leak_file})" + return 1 + else + rm -f "${_leak_file}" + return 0 + fi +} + # don't check dmesg log after test _disable_dmesg_check() { ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 2/5] xfs/122: fix xfs header ordering problems 2018-01-24 23:53 [PATCH 0/5] xfstests: fixes and reflink quota tests Darrick J. Wong 2018-01-24 23:53 ` [PATCH 1/5] common/rc: report kmemleak errors Darrick J. Wong @ 2018-01-24 23:53 ` Darrick J. Wong 2018-01-24 23:53 ` [PATCH 3/5] generic/403: don't spew '$GETFATTR_PROG: Killed' messages Darrick J. Wong ` (2 subsequent siblings) 4 siblings, 0 replies; 11+ messages in thread From: Darrick J. Wong @ 2018-01-24 23:53 UTC (permalink / raw) To: eguan, darrick.wong; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> XFS headers are supposed to be included in a certain order so that inline functions actually compile correctly. For the most part the shell feeds us the files in an order that works, but with the addition of the xfs_dir2_dirblock_bytes function this doesn't always work now. Therefore, explicitly #include the headers in the required order. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/xfs/122 | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/xfs/122 b/tests/xfs/122 index 671a77b..d836670 100755 --- a/tests/xfs/122 +++ b/tests/xfs/122 @@ -79,6 +79,11 @@ cat >$cprog <<EOF #define _GNU_SOURCE #include <stdio.h> EOF +# Certain headers must be included in a certain order... +for hdr in xfs.h xfs_types.h xfs_fs.h xfs_arch.h xfs_format.h; do + test -e "/usr/include/xfs/$hdr" && echo "#include <xfs/$hdr>" >> $cprog +done +# ...but be sure to pull in any new headers that might show up. for hdr in /usr/include/xfs/xfs*.h; do echo "#include <$(echo "$hdr" | sed -e 's|/usr/include/||g')>" >> $cprog done ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 3/5] generic/403: don't spew '$GETFATTR_PROG: Killed' messages 2018-01-24 23:53 [PATCH 0/5] xfstests: fixes and reflink quota tests Darrick J. Wong 2018-01-24 23:53 ` [PATCH 1/5] common/rc: report kmemleak errors Darrick J. Wong 2018-01-24 23:53 ` [PATCH 2/5] xfs/122: fix xfs header ordering problems Darrick J. Wong @ 2018-01-24 23:53 ` Darrick J. Wong 2018-01-25 7:58 ` Eryu Guan 2018-01-24 23:53 ` [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y Darrick J. Wong 2018-01-24 23:53 ` [PATCH 5/5] xfs: regression tests for reflink quota bugs Darrick J. Wong 4 siblings, 1 reply; 11+ messages in thread From: Darrick J. Wong @ 2018-01-24 23:53 UTC (permalink / raw) To: eguan, darrick.wong; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> Use a runfile presence check to control the background getfattr loop instead of using kill -9. This helps us to avoid the problem that the controlling bash will print a process killed message, which wrecks the golden output. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/generic/403 | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/generic/403 b/tests/generic/403 index 17897ff..18fd1e5 100755 --- a/tests/generic/403 +++ b/tests/generic/403 @@ -61,7 +61,9 @@ touch $SCRATCH_MNT/file $SETFATTR_PROG -n trusted.small -v a $SCRATCH_MNT/file # start a background getxattr loop for the existing xattr -while [ true ]; do +runfile="$tmp.getfattr" +touch $runfile +while [ -e $runfile ]; do $GETFATTR_PROG --absolute-names -n trusted.small $SCRATCH_MNT/file \ > /dev/null || break done & @@ -75,7 +77,7 @@ for i in $(seq 0 99); do $SETFATTR_PROG -x trusted.big $SCRATCH_MNT/file done -kill -9 $getfattr_pid > /dev/null 2>&1 +rm -rf $runfile wait > /dev/null 2>&1 echo Silence is golden ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 3/5] generic/403: don't spew '$GETFATTR_PROG: Killed' messages 2018-01-24 23:53 ` [PATCH 3/5] generic/403: don't spew '$GETFATTR_PROG: Killed' messages Darrick J. Wong @ 2018-01-25 7:58 ` Eryu Guan 0 siblings, 0 replies; 11+ messages in thread From: Eryu Guan @ 2018-01-25 7:58 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs, fstests On Wed, Jan 24, 2018 at 03:53:35PM -0800, Darrick J. Wong wrote: > From: Darrick J. Wong <darrick.wong@oracle.com> > > Use a runfile presence check to control the background getfattr loop > instead of using kill -9. This helps us to avoid the problem that > the controlling bash will print a process killed message, which wrecks > the golden output. > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> I saw this failure in automatic fstests runs occasionally, but could never reproduce it manually. Thanks for fixing it! > --- > tests/generic/403 | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > > diff --git a/tests/generic/403 b/tests/generic/403 > index 17897ff..18fd1e5 100755 > --- a/tests/generic/403 > +++ b/tests/generic/403 > @@ -61,7 +61,9 @@ touch $SCRATCH_MNT/file > $SETFATTR_PROG -n trusted.small -v a $SCRATCH_MNT/file > > # start a background getxattr loop for the existing xattr > -while [ true ]; do > +runfile="$tmp.getfattr" > +touch $runfile > +while [ -e $runfile ]; do > $GETFATTR_PROG --absolute-names -n trusted.small $SCRATCH_MNT/file \ > > /dev/null || break > done & > @@ -75,7 +77,7 @@ for i in $(seq 0 99); do > $SETFATTR_PROG -x trusted.big $SCRATCH_MNT/file > done > > -kill -9 $getfattr_pid > /dev/null 2>&1 > +rm -rf $runfile I changed it to 'rm -f $runfile', dropped '-r'. Thanks, Eryu > wait > /dev/null 2>&1 > > echo Silence is golden > ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y 2018-01-24 23:53 [PATCH 0/5] xfstests: fixes and reflink quota tests Darrick J. Wong ` (2 preceding siblings ...) 2018-01-24 23:53 ` [PATCH 3/5] generic/403: don't spew '$GETFATTR_PROG: Killed' messages Darrick J. Wong @ 2018-01-24 23:53 ` Darrick J. Wong 2018-01-25 8:17 ` Eryu Guan 2018-01-24 23:53 ` [PATCH 5/5] xfs: regression tests for reflink quota bugs Darrick J. Wong 4 siblings, 1 reply; 11+ messages in thread From: Darrick J. Wong @ 2018-01-24 23:53 UTC (permalink / raw) To: eguan, darrick.wong; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> These tests requires XFS debugging functionality, so test for that. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/xfs/243 | 2 ++ tests/xfs/245 | 2 ++ tests/xfs/246 | 2 ++ 3 files changed, 6 insertions(+) diff --git a/tests/xfs/243 b/tests/xfs/243 index 0bb69c4..1e6ab5a 100755 --- a/tests/xfs/243 +++ b/tests/xfs/243 @@ -49,10 +49,12 @@ _cleanup() . ./common/rc . ./common/filter . ./common/reflink +. ./common/xfs # real QA test starts here _supported_os Linux _supported_fs xfs +_require_xfs_debug _require_scratch_reflink _require_xfs_io_command "falloc" _require_xfs_io_command "cowextsize" diff --git a/tests/xfs/245 b/tests/xfs/245 index c668eeb..d54f347 100755 --- a/tests/xfs/245 +++ b/tests/xfs/245 @@ -43,10 +43,12 @@ _cleanup() . ./common/rc . ./common/filter . ./common/reflink +. ./common/xfs # real QA test starts here _supported_os Linux _supported_fs xfs +_require_xfs_debug _require_scratch_reflink _require_xfs_io_command "falloc" _require_xfs_io_command "bmap" "-c" diff --git a/tests/xfs/246 b/tests/xfs/246 index 61d2545..c6ab603 100755 --- a/tests/xfs/246 +++ b/tests/xfs/246 @@ -38,10 +38,12 @@ _cleanup() # get standard environment, filters and checks . ./common/rc . ./common/filter +. ./common/xfs # real QA test starts here _supported_os Linux _supported_fs xfs +_require_xfs_debug _require_xfs_io_command "bmap" "-c" _require_scratch ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y 2018-01-24 23:53 ` [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y Darrick J. Wong @ 2018-01-25 8:17 ` Eryu Guan 2018-01-26 0:11 ` Darrick J. Wong 0 siblings, 1 reply; 11+ messages in thread From: Eryu Guan @ 2018-01-25 8:17 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs, fstests On Wed, Jan 24, 2018 at 03:53:41PM -0800, Darrick J. Wong wrote: > From: Darrick J. Wong <darrick.wong@oracle.com> > > These tests requires XFS debugging functionality, so test for that. > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> > --- > tests/xfs/243 | 2 ++ > tests/xfs/245 | 2 ++ > tests/xfs/246 | 2 ++ > 3 files changed, 6 insertions(+) > > > diff --git a/tests/xfs/243 b/tests/xfs/243 > index 0bb69c4..1e6ab5a 100755 > --- a/tests/xfs/243 > +++ b/tests/xfs/243 > @@ -49,10 +49,12 @@ _cleanup() > . ./common/rc > . ./common/filter > . ./common/reflink > +. ./common/xfs This file should be included already by common/rc based on $FSTYP, no need to source it again in test. I'd remove it from all three tests on commit. Thanks, Eryu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y 2018-01-25 8:17 ` Eryu Guan @ 2018-01-26 0:11 ` Darrick J. Wong 0 siblings, 0 replies; 11+ messages in thread From: Darrick J. Wong @ 2018-01-26 0:11 UTC (permalink / raw) To: Eryu Guan; +Cc: linux-xfs, fstests On Thu, Jan 25, 2018 at 04:17:53PM +0800, Eryu Guan wrote: > On Wed, Jan 24, 2018 at 03:53:41PM -0800, Darrick J. Wong wrote: > > From: Darrick J. Wong <darrick.wong@oracle.com> > > > > These tests requires XFS debugging functionality, so test for that. > > > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> > > --- > > tests/xfs/243 | 2 ++ > > tests/xfs/245 | 2 ++ > > tests/xfs/246 | 2 ++ > > 3 files changed, 6 insertions(+) > > > > > > diff --git a/tests/xfs/243 b/tests/xfs/243 > > index 0bb69c4..1e6ab5a 100755 > > --- a/tests/xfs/243 > > +++ b/tests/xfs/243 > > @@ -49,10 +49,12 @@ _cleanup() > > . ./common/rc > > . ./common/filter > > . ./common/reflink > > +. ./common/xfs > > This file should be included already by common/rc based on $FSTYP, no > need to source it again in test. I'd remove it from all three tests on > commit. Ok. Sorry I missed that subtlety. --D > Thanks, > Eryu ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 5/5] xfs: regression tests for reflink quota bugs 2018-01-24 23:53 [PATCH 0/5] xfstests: fixes and reflink quota tests Darrick J. Wong ` (3 preceding siblings ...) 2018-01-24 23:53 ` [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y Darrick J. Wong @ 2018-01-24 23:53 ` Darrick J. Wong 2018-01-26 6:49 ` Eryu Guan 4 siblings, 1 reply; 11+ messages in thread From: Darrick J. Wong @ 2018-01-24 23:53 UTC (permalink / raw) To: eguan, darrick.wong; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> Add three tests to look for quota bugs in xfs reflink. The first test looks for problems when we have speculative cow reservations in memory, we chown the file, but the reservations don't move to the new owner. The second test checks that we remembered to dqattach the inodes before performing reflink operations; and the third exercises reflink quota handling near enospc. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/xfs/904 | 93 ++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/904.out | 17 ++++++++ tests/xfs/905 | 88 ++++++++++++++++++++++++++++++++++++++++++ tests/xfs/905.out | 13 ++++++ tests/xfs/906 | 111 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/906.out | 6 +++ tests/xfs/group | 3 + 7 files changed, 331 insertions(+) create mode 100755 tests/xfs/904 create mode 100644 tests/xfs/904.out create mode 100755 tests/xfs/905 create mode 100644 tests/xfs/905.out create mode 100755 tests/xfs/906 create mode 100644 tests/xfs/906.out diff --git a/tests/xfs/904 b/tests/xfs/904 new file mode 100755 index 0000000..ef78f1a --- /dev/null +++ b/tests/xfs/904 @@ -0,0 +1,93 @@ +#! /bin/bash +# FS QA Test No. 904 +# +# Regression test for a quota accounting bug when changing the owner of +# a file that has CoW reservations and no dirty pages. The reservations +# should shift over to the new owner, but they do not. +# +#----------------------------------------------------------------------- +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/reflink +. ./common/quota +. ./common/filter + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux + +_require_quota +_require_scratch_reflink +_require_cp_reflink +_require_user + +rm -f $seqres.full + +echo "Format and mount" +_scratch_mkfs > "$seqres.full" 2>&1 +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 + +echo "Create files" +$XFS_IO_PROG -c "cowextsize 1m" $SCRATCH_MNT +touch $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa +chown $qa_user $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa +_pwrite_byte 0x58 0 64k $SCRATCH_MNT/a >> $seqres.full +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full +_report_quota_blocks "-u $SCRATCH_MNT" + +echo "Reflink and CoW" +_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/b +_pwrite_byte 0x59 0 64k $SCRATCH_MNT/a >> $seqres.full +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full +_report_quota_blocks "-u $SCRATCH_MNT" + +echo "Sync" +sync +_report_quota_blocks "-u $SCRATCH_MNT" + +echo "Chown and check quota" +chown root $SCRATCH_MNT/a +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full +_report_quota_blocks "-u $SCRATCH_MNT" + +echo "Remount" +_scratch_unmount +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full +_report_quota_blocks "-u $SCRATCH_MNT" + +# success, all done +status=0 +exit diff --git a/tests/xfs/904.out b/tests/xfs/904.out new file mode 100644 index 0000000..4c2f508 --- /dev/null +++ b/tests/xfs/904.out @@ -0,0 +1,17 @@ +QA output created by 904 +Format and mount +Create files +root 0 0 0 +fsgqa 64 0 0 +Reflink and CoW +root 0 0 0 +fsgqa 1152 0 0 +Sync +root 0 0 0 +fsgqa 1088 0 0 +Chown and check quota +root 1024 0 0 +fsgqa 64 0 0 +Remount +root 64 0 0 +fsgqa 64 0 0 diff --git a/tests/xfs/905 b/tests/xfs/905 new file mode 100755 index 0000000..a457c84 --- /dev/null +++ b/tests/xfs/905 @@ -0,0 +1,88 @@ +#! /bin/bash +# FS QA Test No. 905 +# +# Regression test for a quota accounting bug when reflinking across EOF +# of a file in which we forgot dq_attach. +# +#----------------------------------------------------------------------- +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/reflink +. ./common/quota +. ./common/filter + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux + +_require_quota +_require_scratch_reflink +_require_cp_reflink +_require_user + +rm -f $seqres.full + +check_quota() { + du_total="$(du -ksc $SCRATCH_MNT/a $SCRATCH_MNT/b | tail -n 1 | awk '{print $1}')" + qu_total="$(_report_quota_blocks "-u $SCRATCH_MNT" | grep $qa_user | awk '{print $2}')" + echo "du: $du_total; quota: $qu_total" +} + +echo "Format and mount (noquota)" +_scratch_mkfs > "$seqres.full" 2>&1 +_scratch_mount "-o noquota" >> $seqres.full 2>&1 + +echo "Create files" +_pwrite_byte 0x58 0 1m $SCRATCH_MNT/a >> $seqres.full +_pwrite_byte 0x58 0 1m $SCRATCH_MNT/b >> $seqres.full +chown $qa_user $SCRATCH_MNT/a $SCRATCH_MNT/b +check_quota 2>&1 | _filter_scratch + +echo "Mount (usrquota) and check quota" +_scratch_unmount +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 +check_quota + +echo "Reflink and check quota again" +_reflink_range $SCRATCH_MNT/a 256k $SCRATCH_MNT/b 384k 128k >> $seqres.full +_reflink_range $SCRATCH_MNT/a 256k $SCRATCH_MNT/b 960k 128k >> $seqres.full +check_quota + +echo "Force quotacheck" +_check_quota_usage + +# success, all done +status=0 +exit diff --git a/tests/xfs/905.out b/tests/xfs/905.out new file mode 100644 index 0000000..fa95624 --- /dev/null +++ b/tests/xfs/905.out @@ -0,0 +1,13 @@ +QA output created by 905 +Format and mount (noquota) +Create files +repquota: Mountpoint (or device) SCRATCH_MNT not found or has no quota enabled. +repquota: Not all specified mountpoints are using quota. +du: 2048; quota: +Mount (usrquota) and check quota +du: 2048; quota: 2048 +Reflink and check quota again +du: 2112; quota: 2112 +Force quotacheck +Comparing user usage +Comparing group usage diff --git a/tests/xfs/906 b/tests/xfs/906 new file mode 100755 index 0000000..17dcd2e --- /dev/null +++ b/tests/xfs/906 @@ -0,0 +1,111 @@ +#! /bin/bash +# FS QA Test No. 906 +# +# Force enable all XFS quotas, run fsstress until the fs runs out of +# space, and make sure the quotas are still correct when we're done. +# This is a general regression/stress test for numerous quota bugs with +# reflink and copy on write. +# +#----------------------------------------------------------------------- +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* + $KILLALL_PROG -9 fsstress > /dev/null 2>&1 +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/quota +. ./common/filter +. ./common/reflink + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux + +_require_scratch_reflink +_require_quota +_require_command "$KILLALL_PROG" "killall" + +rm -f $seqres.full + +report_quota_blocks() { + $XFS_QUOTA_PROG -x -c "report $1" $SCRATCH_MNT | \ + awk '{x += $2;} END { print(x); }' +} + +compare_quota_to_du() { + test $1 -eq $2 || echo "$3 quota $2 blocks does not match du $1 blocks?" +} + +# Make sure the user/group/project quota block counts match the du output. +# This ensures that we did the quota accounting correctly and that we're +# accurately reporting cow preallocation blocks in stat. +check_quota_du_blocks() { + sync + $XFS_QUOTA_PROG -x -c 'report' $SCRATCH_MNT >> $seqres.full + du_rep=$(du -ks $SCRATCH_MNT | awk '{print $1}') + u_rep=$(report_quota_blocks -u) + g_rep=$(report_quota_blocks -g) + p_rep=$(report_quota_blocks -p) + + compare_quota_to_du $du_rep $u_rep "user" + compare_quota_to_du $du_rep $g_rep "group" + compare_quota_to_du $du_rep $p_rep "project" +} + +echo "Format and fsstress" + +_qmount_option "usrquota,grpquota,prjquota" +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +nr_cpus=$((LOAD_FACTOR * 4)) +nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) +$FSSTRESS_PROG $FSSTRESS_AVOID -w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus >> $seqres.full + +echo "Check quota before remount" +check_quota_du_blocks + +# Clear out all the preallocations before we quotacheck. +# The count comparison in _check_quota_usage will be unhappy if we don't +# manage to clean out all the cow preallocations before the remount. +_scratch_unmount +_scratch_mount + +# Make sure the usage doesn't change after quotacheck. +echo "Check quota after remount" +_check_quota_usage + +check_quota_du_blocks + +# success, all done +status=0 +exit diff --git a/tests/xfs/906.out b/tests/xfs/906.out new file mode 100644 index 0000000..201daf3 --- /dev/null +++ b/tests/xfs/906.out @@ -0,0 +1,6 @@ +QA output created by 906 +Format and fsstress +Check quota before remount +Check quota after remount +Comparing user usage +Comparing group usage diff --git a/tests/xfs/group b/tests/xfs/group index cf81451..aa8b49f 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -437,3 +437,6 @@ 437 auto quick other 438 auto quick quota dangerous 439 auto quick fuzzers log +904 auto quick clone quota +905 auto quick clone quota +906 auto quick clone quota ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 5/5] xfs: regression tests for reflink quota bugs 2018-01-24 23:53 ` [PATCH 5/5] xfs: regression tests for reflink quota bugs Darrick J. Wong @ 2018-01-26 6:49 ` Eryu Guan 2018-01-26 7:00 ` Darrick J. Wong 0 siblings, 1 reply; 11+ messages in thread From: Eryu Guan @ 2018-01-26 6:49 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs, fstests On Wed, Jan 24, 2018 at 03:53:47PM -0800, Darrick J. Wong wrote: > From: Darrick J. Wong <darrick.wong@oracle.com> > > Add three tests to look for quota bugs in xfs reflink. The first > test looks for problems when we have speculative cow reservations > in memory, we chown the file, but the reservations don't move to > the new owner. The second test checks that we remembered to > dqattach the inodes before performing reflink operations; and the > third exercises reflink quota handling near enospc. > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> > --- > tests/xfs/904 | 93 ++++++++++++++++++++++++++++++++++++++++++++ > tests/xfs/904.out | 17 ++++++++ > tests/xfs/905 | 88 ++++++++++++++++++++++++++++++++++++++++++ > tests/xfs/905.out | 13 ++++++ > tests/xfs/906 | 111 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > tests/xfs/906.out | 6 +++ > tests/xfs/group | 3 + > 7 files changed, 331 insertions(+) > create mode 100755 tests/xfs/904 > create mode 100644 tests/xfs/904.out > create mode 100755 tests/xfs/905 > create mode 100644 tests/xfs/905.out > create mode 100755 tests/xfs/906 > create mode 100644 tests/xfs/906.out > > > diff --git a/tests/xfs/904 b/tests/xfs/904 > new file mode 100755 > index 0000000..ef78f1a > --- /dev/null > +++ b/tests/xfs/904 > @@ -0,0 +1,93 @@ > +#! /bin/bash > +# FS QA Test No. 904 > +# > +# Regression test for a quota accounting bug when changing the owner of > +# a file that has CoW reservations and no dirty pages. The reservations > +# should shift over to the new owner, but they do not. > +# > +#----------------------------------------------------------------------- > +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/reflink > +. ./common/quota > +. ./common/filter > + > +# Modify as appropriate. > +_supported_fs generic > +_supported_os Linux > + > +_require_quota > +_require_scratch_reflink > +_require_cp_reflink > +_require_user > + > +rm -f $seqres.full > + > +echo "Format and mount" > +_scratch_mkfs > "$seqres.full" 2>&1 > +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 > + > +echo "Create files" > +$XFS_IO_PROG -c "cowextsize 1m" $SCRATCH_MNT Add comments on why we need this? > +touch $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa > +chown $qa_user $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa > +_pwrite_byte 0x58 0 64k $SCRATCH_MNT/a >> $seqres.full > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > +_report_quota_blocks "-u $SCRATCH_MNT" > + > +echo "Reflink and CoW" > +_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/b > +_pwrite_byte 0x59 0 64k $SCRATCH_MNT/a >> $seqres.full > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > +_report_quota_blocks "-u $SCRATCH_MNT" > + > +echo "Sync" > +sync > +_report_quota_blocks "-u $SCRATCH_MNT" > + > +echo "Chown and check quota" > +chown root $SCRATCH_MNT/a > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > +_report_quota_blocks "-u $SCRATCH_MNT" > + > +echo "Remount" > +_scratch_unmount > +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > +_report_quota_blocks "-u $SCRATCH_MNT" > + > +# success, all done > +status=0 > +exit > diff --git a/tests/xfs/904.out b/tests/xfs/904.out > new file mode 100644 > index 0000000..4c2f508 > --- /dev/null > +++ b/tests/xfs/904.out > @@ -0,0 +1,17 @@ > +QA output created by 904 > +Format and mount > +Create files > +root 0 0 0 > +fsgqa 64 0 0 > +Reflink and CoW > +root 0 0 0 > +fsgqa 1152 0 0 > +Sync > +root 0 0 0 > +fsgqa 1088 0 0 > +Chown and check quota > +root 1024 0 0 > +fsgqa 64 0 0 > +Remount > +root 64 0 0 > +fsgqa 64 0 0 > diff --git a/tests/xfs/905 b/tests/xfs/905 > new file mode 100755 > index 0000000..a457c84 > --- /dev/null > +++ b/tests/xfs/905 > @@ -0,0 +1,88 @@ > +#! /bin/bash > +# FS QA Test No. 905 > +# > +# Regression test for a quota accounting bug when reflinking across EOF > +# of a file in which we forgot dq_attach. > +# > +#----------------------------------------------------------------------- > +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/reflink > +. ./common/quota > +. ./common/filter > + > +# Modify as appropriate. > +_supported_fs generic > +_supported_os Linux > + > +_require_quota > +_require_scratch_reflink > +_require_cp_reflink > +_require_user > + > +rm -f $seqres.full > + > +check_quota() { > + du_total="$(du -ksc $SCRATCH_MNT/a $SCRATCH_MNT/b | tail -n 1 | awk '{print $1}')" > + qu_total="$(_report_quota_blocks "-u $SCRATCH_MNT" | grep $qa_user | awk '{print $2}')" > + echo "du: $du_total; quota: $qu_total" > +} > + > +echo "Format and mount (noquota)" > +_scratch_mkfs > "$seqres.full" 2>&1 > +_scratch_mount "-o noquota" >> $seqres.full 2>&1 > + > +echo "Create files" > +_pwrite_byte 0x58 0 1m $SCRATCH_MNT/a >> $seqres.full > +_pwrite_byte 0x58 0 1m $SCRATCH_MNT/b >> $seqres.full > +chown $qa_user $SCRATCH_MNT/a $SCRATCH_MNT/b > +check_quota 2>&1 | _filter_scratch > + > +echo "Mount (usrquota) and check quota" > +_scratch_unmount > +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 > +check_quota > + > +echo "Reflink and check quota again" > +_reflink_range $SCRATCH_MNT/a 256k $SCRATCH_MNT/b 384k 128k >> $seqres.full > +_reflink_range $SCRATCH_MNT/a 256k $SCRATCH_MNT/b 960k 128k >> $seqres.full > +check_quota > + > +echo "Force quotacheck" > +_check_quota_usage > + > +# success, all done > +status=0 > +exit > diff --git a/tests/xfs/905.out b/tests/xfs/905.out > new file mode 100644 > index 0000000..fa95624 > --- /dev/null > +++ b/tests/xfs/905.out > @@ -0,0 +1,13 @@ > +QA output created by 905 > +Format and mount (noquota) > +Create files > +repquota: Mountpoint (or device) SCRATCH_MNT not found or has no quota enabled. > +repquota: Not all specified mountpoints are using quota. > +du: 2048; quota: > +Mount (usrquota) and check quota > +du: 2048; quota: 2048 > +Reflink and check quota again > +du: 2112; quota: 2112 > +Force quotacheck > +Comparing user usage > +Comparing group usage > diff --git a/tests/xfs/906 b/tests/xfs/906 > new file mode 100755 > index 0000000..17dcd2e > --- /dev/null > +++ b/tests/xfs/906 > @@ -0,0 +1,111 @@ > +#! /bin/bash > +# FS QA Test No. 906 > +# > +# Force enable all XFS quotas, run fsstress until the fs runs out of > +# space, and make sure the quotas are still correct when we're done. > +# This is a general regression/stress test for numerous quota bugs with > +# reflink and copy on write. > +# > +#----------------------------------------------------------------------- > +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > + $KILLALL_PROG -9 fsstress > /dev/null 2>&1 > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/quota > +. ./common/filter > +. ./common/reflink > + > +# Modify as appropriate. > +_supported_fs generic > +_supported_os Linux > + > +_require_scratch_reflink > +_require_quota > +_require_command "$KILLALL_PROG" "killall" > + > +rm -f $seqres.full > + > +report_quota_blocks() { > + $XFS_QUOTA_PROG -x -c "report $1" $SCRATCH_MNT | \ > + awk '{x += $2;} END { print(x); }' > +} > + > +compare_quota_to_du() { > + test $1 -eq $2 || echo "$3 quota $2 blocks does not match du $1 blocks?" > +} > + > +# Make sure the user/group/project quota block counts match the du output. > +# This ensures that we did the quota accounting correctly and that we're > +# accurately reporting cow preallocation blocks in stat. > +check_quota_du_blocks() { > + sync > + $XFS_QUOTA_PROG -x -c 'report' $SCRATCH_MNT >> $seqres.full > + du_rep=$(du -ks $SCRATCH_MNT | awk '{print $1}') > + u_rep=$(report_quota_blocks -u) > + g_rep=$(report_quota_blocks -g) > + p_rep=$(report_quota_blocks -p) > + > + compare_quota_to_du $du_rep $u_rep "user" > + compare_quota_to_du $du_rep $g_rep "group" > + compare_quota_to_du $du_rep $p_rep "project" > +} > + > +echo "Format and fsstress" > + > +_qmount_option "usrquota,grpquota,prjquota" > +_scratch_mkfs > $seqres.full 2>&1 Do we need to limit the fs size? If we're running on a large SCRATCH_DEV test will run for longer time. > +_scratch_mount >> $seqres.full 2>&1 I'd check mount status here and _fail if mount failed, otherwise we may eat all space on underlying fs (usually rootfs). > + > +nr_cpus=$((LOAD_FACTOR * 4)) > +nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) > +$FSSTRESS_PROG $FSSTRESS_AVOID -w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus >> $seqres.full > + > +echo "Check quota before remount" > +check_quota_du_blocks > + > +# Clear out all the preallocations before we quotacheck. > +# The count comparison in _check_quota_usage will be unhappy if we don't > +# manage to clean out all the cow preallocations before the remount. > +_scratch_unmount > +_scratch_mount > + > +# Make sure the usage doesn't change after quotacheck. > +echo "Check quota after remount" > +_check_quota_usage > + > +check_quota_du_blocks > + > +# success, all done > +status=0 > +exit > diff --git a/tests/xfs/906.out b/tests/xfs/906.out > new file mode 100644 > index 0000000..201daf3 > --- /dev/null > +++ b/tests/xfs/906.out > @@ -0,0 +1,6 @@ > +QA output created by 906 > +Format and fsstress > +Check quota before remount > +Check quota after remount > +Comparing user usage > +Comparing group usage > diff --git a/tests/xfs/group b/tests/xfs/group > index cf81451..aa8b49f 100644 > --- a/tests/xfs/group > +++ b/tests/xfs/group > @@ -437,3 +437,6 @@ > 437 auto quick other > 438 auto quick quota dangerous > 439 auto quick fuzzers log > +904 auto quick clone quota > +905 auto quick clone quota > +906 auto quick clone quota I think 906 can be a 'stress' test instead of 'quick'. Thanks, Eryu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 5/5] xfs: regression tests for reflink quota bugs 2018-01-26 6:49 ` Eryu Guan @ 2018-01-26 7:00 ` Darrick J. Wong 0 siblings, 0 replies; 11+ messages in thread From: Darrick J. Wong @ 2018-01-26 7:00 UTC (permalink / raw) To: Eryu Guan; +Cc: linux-xfs, fstests On Fri, Jan 26, 2018 at 02:49:52PM +0800, Eryu Guan wrote: > On Wed, Jan 24, 2018 at 03:53:47PM -0800, Darrick J. Wong wrote: > > From: Darrick J. Wong <darrick.wong@oracle.com> > > > > Add three tests to look for quota bugs in xfs reflink. The first > > test looks for problems when we have speculative cow reservations > > in memory, we chown the file, but the reservations don't move to > > the new owner. The second test checks that we remembered to > > dqattach the inodes before performing reflink operations; and the > > third exercises reflink quota handling near enospc. > > > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> > > --- > > tests/xfs/904 | 93 ++++++++++++++++++++++++++++++++++++++++++++ > > tests/xfs/904.out | 17 ++++++++ > > tests/xfs/905 | 88 ++++++++++++++++++++++++++++++++++++++++++ > > tests/xfs/905.out | 13 ++++++ > > tests/xfs/906 | 111 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > > tests/xfs/906.out | 6 +++ > > tests/xfs/group | 3 + > > 7 files changed, 331 insertions(+) > > create mode 100755 tests/xfs/904 > > create mode 100644 tests/xfs/904.out > > create mode 100755 tests/xfs/905 > > create mode 100644 tests/xfs/905.out > > create mode 100755 tests/xfs/906 > > create mode 100644 tests/xfs/906.out > > > > > > diff --git a/tests/xfs/904 b/tests/xfs/904 > > new file mode 100755 > > index 0000000..ef78f1a > > --- /dev/null > > +++ b/tests/xfs/904 > > @@ -0,0 +1,93 @@ > > +#! /bin/bash > > +# FS QA Test No. 904 > > +# > > +# Regression test for a quota accounting bug when changing the owner of > > +# a file that has CoW reservations and no dirty pages. The reservations > > +# should shift over to the new owner, but they do not. > > +# > > +#----------------------------------------------------------------------- > > +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. > > +# > > +# This program is free software; you can redistribute it and/or > > +# modify it under the terms of the GNU General Public License as > > +# published by the Free Software Foundation. > > +# > > +# This program is distributed in the hope that it would be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program; if not, write the Free Software Foundation, > > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > > +#----------------------------------------------------------------------- > > +# > > + > > +seq=`basename $0` > > +seqres=$RESULT_DIR/$seq > > +echo "QA output created by $seq" > > + > > +here=`pwd` > > +tmp=/tmp/$$ > > +status=1 # failure is the default! > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > + > > +_cleanup() > > +{ > > + cd / > > + rm -f $tmp.* > > +} > > + > > +# get standard environment, filters and checks > > +. ./common/rc > > +. ./common/reflink > > +. ./common/quota > > +. ./common/filter > > + > > +# Modify as appropriate. > > +_supported_fs generic > > +_supported_os Linux > > + > > +_require_quota > > +_require_scratch_reflink > > +_require_cp_reflink > > +_require_user > > + > > +rm -f $seqres.full > > + > > +echo "Format and mount" > > +_scratch_mkfs > "$seqres.full" 2>&1 > > +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 > > + > > +echo "Create files" > > +$XFS_IO_PROG -c "cowextsize 1m" $SCRATCH_MNT > > Add comments on why we need this? To make sure there are a bunch of cow block reservations sitting around when we chown even after the write flushes and remaps. > > +touch $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa > > +chown $qa_user $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa > > +_pwrite_byte 0x58 0 64k $SCRATCH_MNT/a >> $seqres.full > > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > > +_report_quota_blocks "-u $SCRATCH_MNT" > > + > > +echo "Reflink and CoW" > > +_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/b > > +_pwrite_byte 0x59 0 64k $SCRATCH_MNT/a >> $seqres.full > > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > > +_report_quota_blocks "-u $SCRATCH_MNT" > > + > > +echo "Sync" > > +sync > > +_report_quota_blocks "-u $SCRATCH_MNT" > > + > > +echo "Chown and check quota" > > +chown root $SCRATCH_MNT/a > > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > > +_report_quota_blocks "-u $SCRATCH_MNT" > > + > > +echo "Remount" > > +_scratch_unmount > > +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 > > +$XFS_IO_PROG -c 'stat -r' $SCRATCH_MNT/a | grep stat.size >> $seqres.full > > +_report_quota_blocks "-u $SCRATCH_MNT" > > + > > +# success, all done > > +status=0 > > +exit > > diff --git a/tests/xfs/904.out b/tests/xfs/904.out > > new file mode 100644 > > index 0000000..4c2f508 > > --- /dev/null > > +++ b/tests/xfs/904.out > > @@ -0,0 +1,17 @@ > > +QA output created by 904 > > +Format and mount > > +Create files > > +root 0 0 0 > > +fsgqa 64 0 0 > > +Reflink and CoW > > +root 0 0 0 > > +fsgqa 1152 0 0 > > +Sync > > +root 0 0 0 > > +fsgqa 1088 0 0 > > +Chown and check quota > > +root 1024 0 0 > > +fsgqa 64 0 0 > > +Remount > > +root 64 0 0 > > +fsgqa 64 0 0 > > diff --git a/tests/xfs/905 b/tests/xfs/905 > > new file mode 100755 > > index 0000000..a457c84 > > --- /dev/null > > +++ b/tests/xfs/905 > > @@ -0,0 +1,88 @@ > > +#! /bin/bash > > +# FS QA Test No. 905 > > +# > > +# Regression test for a quota accounting bug when reflinking across EOF > > +# of a file in which we forgot dq_attach. > > +# > > +#----------------------------------------------------------------------- > > +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. > > +# > > +# This program is free software; you can redistribute it and/or > > +# modify it under the terms of the GNU General Public License as > > +# published by the Free Software Foundation. > > +# > > +# This program is distributed in the hope that it would be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program; if not, write the Free Software Foundation, > > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > > +#----------------------------------------------------------------------- > > +# > > + > > +seq=`basename $0` > > +seqres=$RESULT_DIR/$seq > > +echo "QA output created by $seq" > > + > > +here=`pwd` > > +tmp=/tmp/$$ > > +status=1 # failure is the default! > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > + > > +_cleanup() > > +{ > > + cd / > > + rm -f $tmp.* > > +} > > + > > +# get standard environment, filters and checks > > +. ./common/rc > > +. ./common/reflink > > +. ./common/quota > > +. ./common/filter > > + > > +# Modify as appropriate. > > +_supported_fs generic > > +_supported_os Linux > > + > > +_require_quota > > +_require_scratch_reflink > > +_require_cp_reflink > > +_require_user > > + > > +rm -f $seqres.full > > + > > +check_quota() { > > + du_total="$(du -ksc $SCRATCH_MNT/a $SCRATCH_MNT/b | tail -n 1 | awk '{print $1}')" > > + qu_total="$(_report_quota_blocks "-u $SCRATCH_MNT" | grep $qa_user | awk '{print $2}')" > > + echo "du: $du_total; quota: $qu_total" > > +} > > + > > +echo "Format and mount (noquota)" > > +_scratch_mkfs > "$seqres.full" 2>&1 > > +_scratch_mount "-o noquota" >> $seqres.full 2>&1 > > + > > +echo "Create files" > > +_pwrite_byte 0x58 0 1m $SCRATCH_MNT/a >> $seqres.full > > +_pwrite_byte 0x58 0 1m $SCRATCH_MNT/b >> $seqres.full > > +chown $qa_user $SCRATCH_MNT/a $SCRATCH_MNT/b > > +check_quota 2>&1 | _filter_scratch > > + > > +echo "Mount (usrquota) and check quota" > > +_scratch_unmount > > +_scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1 > > +check_quota > > + > > +echo "Reflink and check quota again" > > +_reflink_range $SCRATCH_MNT/a 256k $SCRATCH_MNT/b 384k 128k >> $seqres.full > > +_reflink_range $SCRATCH_MNT/a 256k $SCRATCH_MNT/b 960k 128k >> $seqres.full > > +check_quota > > + > > +echo "Force quotacheck" > > +_check_quota_usage > > + > > +# success, all done > > +status=0 > > +exit > > diff --git a/tests/xfs/905.out b/tests/xfs/905.out > > new file mode 100644 > > index 0000000..fa95624 > > --- /dev/null > > +++ b/tests/xfs/905.out > > @@ -0,0 +1,13 @@ > > +QA output created by 905 > > +Format and mount (noquota) > > +Create files > > +repquota: Mountpoint (or device) SCRATCH_MNT not found or has no quota enabled. > > +repquota: Not all specified mountpoints are using quota. > > +du: 2048; quota: > > +Mount (usrquota) and check quota > > +du: 2048; quota: 2048 > > +Reflink and check quota again > > +du: 2112; quota: 2112 > > +Force quotacheck > > +Comparing user usage > > +Comparing group usage > > diff --git a/tests/xfs/906 b/tests/xfs/906 > > new file mode 100755 > > index 0000000..17dcd2e > > --- /dev/null > > +++ b/tests/xfs/906 > > @@ -0,0 +1,111 @@ > > +#! /bin/bash > > +# FS QA Test No. 906 > > +# > > +# Force enable all XFS quotas, run fsstress until the fs runs out of > > +# space, and make sure the quotas are still correct when we're done. > > +# This is a general regression/stress test for numerous quota bugs with > > +# reflink and copy on write. > > +# > > +#----------------------------------------------------------------------- > > +# Copyright (c) 2018 Oracle, Inc. All Rights Reserved. > > +# > > +# This program is free software; you can redistribute it and/or > > +# modify it under the terms of the GNU General Public License as > > +# published by the Free Software Foundation. > > +# > > +# This program is distributed in the hope that it would be useful, > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +# GNU General Public License for more details. > > +# > > +# You should have received a copy of the GNU General Public License > > +# along with this program; if not, write the Free Software Foundation, > > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > > +#----------------------------------------------------------------------- > > +# > > + > > +seq=`basename $0` > > +seqres=$RESULT_DIR/$seq > > +echo "QA output created by $seq" > > + > > +here=`pwd` > > +tmp=/tmp/$$ > > +status=1 # failure is the default! > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > + > > +_cleanup() > > +{ > > + cd / > > + rm -f $tmp.* > > + $KILLALL_PROG -9 fsstress > /dev/null 2>&1 > > +} > > + > > +# get standard environment, filters and checks > > +. ./common/rc > > +. ./common/quota > > +. ./common/filter > > +. ./common/reflink > > + > > +# Modify as appropriate. > > +_supported_fs generic > > +_supported_os Linux > > + > > +_require_scratch_reflink > > +_require_quota > > +_require_command "$KILLALL_PROG" "killall" > > + > > +rm -f $seqres.full > > + > > +report_quota_blocks() { > > + $XFS_QUOTA_PROG -x -c "report $1" $SCRATCH_MNT | \ > > + awk '{x += $2;} END { print(x); }' > > +} > > + > > +compare_quota_to_du() { > > + test $1 -eq $2 || echo "$3 quota $2 blocks does not match du $1 blocks?" > > +} > > + > > +# Make sure the user/group/project quota block counts match the du output. > > +# This ensures that we did the quota accounting correctly and that we're > > +# accurately reporting cow preallocation blocks in stat. > > +check_quota_du_blocks() { > > + sync > > + $XFS_QUOTA_PROG -x -c 'report' $SCRATCH_MNT >> $seqres.full > > + du_rep=$(du -ks $SCRATCH_MNT | awk '{print $1}') > > + u_rep=$(report_quota_blocks -u) > > + g_rep=$(report_quota_blocks -g) > > + p_rep=$(report_quota_blocks -p) > > + > > + compare_quota_to_du $du_rep $u_rep "user" > > + compare_quota_to_du $du_rep $g_rep "group" > > + compare_quota_to_du $du_rep $p_rep "project" > > +} > > + > > +echo "Format and fsstress" > > + > > +_qmount_option "usrquota,grpquota,prjquota" > > +_scratch_mkfs > $seqres.full 2>&1 > > Do we need to limit the fs size? If we're running on a large SCRATCH_DEV > test will run for longer time. Yeah, we might as well do that. > > +_scratch_mount >> $seqres.full 2>&1 > > I'd check mount status here and _fail if mount failed, otherwise we may > eat all space on underlying fs (usually rootfs). ok. > > + > > +nr_cpus=$((LOAD_FACTOR * 4)) > > +nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) > > +$FSSTRESS_PROG $FSSTRESS_AVOID -w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus >> $seqres.full > > + > > +echo "Check quota before remount" > > +check_quota_du_blocks > > + > > +# Clear out all the preallocations before we quotacheck. > > +# The count comparison in _check_quota_usage will be unhappy if we don't > > +# manage to clean out all the cow preallocations before the remount. > > +_scratch_unmount > > +_scratch_mount > > + > > +# Make sure the usage doesn't change after quotacheck. > > +echo "Check quota after remount" > > +_check_quota_usage > > + > > +check_quota_du_blocks > > + > > +# success, all done > > +status=0 > > +exit > > diff --git a/tests/xfs/906.out b/tests/xfs/906.out > > new file mode 100644 > > index 0000000..201daf3 > > --- /dev/null > > +++ b/tests/xfs/906.out > > @@ -0,0 +1,6 @@ > > +QA output created by 906 > > +Format and fsstress > > +Check quota before remount > > +Check quota after remount > > +Comparing user usage > > +Comparing group usage > > diff --git a/tests/xfs/group b/tests/xfs/group > > index cf81451..aa8b49f 100644 > > --- a/tests/xfs/group > > +++ b/tests/xfs/group > > @@ -437,3 +437,6 @@ > > 437 auto quick other > > 438 auto quick quota dangerous > > 439 auto quick fuzzers log > > +904 auto quick clone quota > > +905 auto quick clone quota > > +906 auto quick clone quota > > I think 906 can be a 'stress' test instead of 'quick'. I don't think it's quick anymore anyway :) --D > Thanks, > Eryu ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2018-01-26 7:13 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-01-24 23:53 [PATCH 0/5] xfstests: fixes and reflink quota tests Darrick J. Wong 2018-01-24 23:53 ` [PATCH 1/5] common/rc: report kmemleak errors Darrick J. Wong 2018-01-24 23:53 ` [PATCH 2/5] xfs/122: fix xfs header ordering problems Darrick J. Wong 2018-01-24 23:53 ` [PATCH 3/5] generic/403: don't spew '$GETFATTR_PROG: Killed' messages Darrick J. Wong 2018-01-25 7:58 ` Eryu Guan 2018-01-24 23:53 ` [PATCH 4/5] xfs/24[356]: checking cow fork bmap requires CONFIG_XFS_DEBUG=y Darrick J. Wong 2018-01-25 8:17 ` Eryu Guan 2018-01-26 0:11 ` Darrick J. Wong 2018-01-24 23:53 ` [PATCH 5/5] xfs: regression tests for reflink quota bugs Darrick J. Wong 2018-01-26 6:49 ` Eryu Guan 2018-01-26 7:00 ` Darrick J. Wong
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.