Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent

["Darrick J. Wong" <darrick.wong@oracle.com>]

On Mon, Dec 02, 2019 at 11:24:24AM -0800, Omar Sandoval wrote:
> On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote:
> > On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> > > From: Omar Sandoval <osandov@fb.com>
> > > 
> > > The only XFS-specific part of this test is the setup, so we can make the
> > > rest a generic test. It's slow, though, as it needs to write 8GB to
> > > convert a big unwritten extent to written.
> > > 
> > > Signed-off-by: Omar Sandoval <osandov@fb.com>
> > > ---
> > >  tests/generic/586     | 59 +++++++++++++++++++++++++++++++++++++++++++
> > >  tests/generic/586.out |  2 ++
> > >  tests/generic/group   |  1 +
> > >  3 files changed, 62 insertions(+)
> > >  create mode 100755 tests/generic/586
> > >  create mode 100644 tests/generic/586.out
> > > 
> > > diff --git a/tests/generic/586 b/tests/generic/586
> > > new file mode 100755
> > > index 00000000..5bcad68b
> > > --- /dev/null
> > > +++ b/tests/generic/586
> > > @@ -0,0 +1,59 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2019 Facebook.  All Rights Reserved.
> > > +#
> > > +# FS QA Test 586
> > > +#
> > > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> > > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> > > +# forever. On other filesystems, this just tests writing into big fallocates.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1	# failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > +	cd /
> > > +	rm -f $tmp.*
> > > +}
> > > +
> > > +. ./common/rc
> > > +. ./common/filter
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_supported_fs generic
> > > +_supported_os Linux
> > > +_require_scratch
> > > +
> > > +maxextlen=$((0x1fffff))
> > 
> > /me wonders if we ought to move this to common/xfs and hoist the other
> > place (xfs/507) where we define this.
> > 
> > > +bs=4096
> > > +rextsize=4
> > > +
> > > +extra_options=""
> > > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> > > +	extra_options="$extra_options -bsize=$bs"
> > 
> > Hm.  Could you rework this to _require_realtime, and if the caller
> > didn't set SCRATCH_RTDEV, create a file on the test device so that we
> > can always do the realtime test if the kernel supports it?  That will
> > ensure that this gets more testing that it does now...
> 
> Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was
> set. Did you mean something like this?
> 
> if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> 	use the configured rtdev
> else
> 	_require_test
> 	_require_scratch
> 	set up a loop device on the test filesystem as the rtdev
> fi

Yep, that's exactly what I meant.

--D

> > > +	extra_options="$extra_options -r extsize=$((bs * rextsize))"
> > > +	extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
> > 
> > ...particularly because I don't think very many people actually run
> > fstests with rt enabled /and/ rtinherit set to stress the realtime
> > allocator.
> > 
> > (I did a year ago and out came a torrent of bugs such that someone could
> > probably write themselves a nice year end bonus just fixing all that,
> > software is terrible :()
> > 
> > > +fi
> > > +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> > > +
> > > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> > > +sync
> > 
> > $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file
> 
> Will fix.
> 
> > (Hm ok, fallocate the first 2097148 blocks of the file...)
> 
> Oops, I'll add some explanations for all of this stuff.
> 
> Thanks for taking a look!
> 
> > > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> > > +sync
> > 
> > $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file
> > 
> > (now fallocate blocks 2097148 to 2097152)
> > 
> > (Not sure why you do the last rtext separately...)
> > 
> > > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> > > +sync
> > 
> > $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file
> > 
> > (and finally write to block 2097149?)
> > 
> > --D