* [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
@ 2019-11-26 20:13 Omar Sandoval
2019-11-27 1:26 ` Darrick J. Wong
0 siblings, 1 reply; 4+ messages in thread
From: Omar Sandoval @ 2019-11-26 20:13 UTC (permalink / raw)
To: fstests; +Cc: kernel-team, linux-xfs
From: Omar Sandoval <osandov@fb.com>
The only XFS-specific part of this test is the setup, so we can make the
rest a generic test. It's slow, though, as it needs to write 8GB to
convert a big unwritten extent to written.
Signed-off-by: Omar Sandoval <osandov@fb.com>
---
tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/586.out | 2 ++
tests/generic/group | 1 +
3 files changed, 62 insertions(+)
create mode 100755 tests/generic/586
create mode 100644 tests/generic/586.out
diff --git a/tests/generic/586 b/tests/generic/586
new file mode 100755
index 00000000..5bcad68b
--- /dev/null
+++ b/tests/generic/586
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Facebook. All Rights Reserved.
+#
+# FS QA Test 586
+#
+# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
+# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
+# forever. On other filesystems, this just tests writing into big fallocates.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+. ./common/rc
+. ./common/filter
+
+rm -f $seqres.full
+
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+maxextlen=$((0x1fffff))
+bs=4096
+rextsize=4
+
+extra_options=""
+if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
+ extra_options="$extra_options -bsize=$bs"
+ extra_options="$extra_options -r extsize=$((bs * rextsize))"
+ extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
+fi
+_scratch_mkfs $extra_options >>$seqres.full 2>&1
+_scratch_mount
+_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
+
+fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
+sync
+fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
+sync
+dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
+sync
+rm "$SCRATCH_MNT/file"
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/generic/586.out b/tests/generic/586.out
new file mode 100644
index 00000000..3d36442d
--- /dev/null
+++ b/tests/generic/586.out
@@ -0,0 +1,2 @@
+QA output created by 586
+Silence is golden
diff --git a/tests/generic/group b/tests/generic/group
index e5d0c1da..6ddaf640 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -588,3 +588,4 @@
583 auto quick encrypt
584 auto quick encrypt
585 auto rename
+586 auto prealloc preallocrw dangerous
--
2.24.0
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
2019-11-26 20:13 [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent Omar Sandoval
@ 2019-11-27 1:26 ` Darrick J. Wong
2019-12-02 19:24 ` Omar Sandoval
0 siblings, 1 reply; 4+ messages in thread
From: Darrick J. Wong @ 2019-11-27 1:26 UTC (permalink / raw)
To: Omar Sandoval; +Cc: fstests, kernel-team, linux-xfs
On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
>
> The only XFS-specific part of this test is the setup, so we can make the
> rest a generic test. It's slow, though, as it needs to write 8GB to
> convert a big unwritten extent to written.
>
> Signed-off-by: Omar Sandoval <osandov@fb.com>
> ---
> tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/586.out | 2 ++
> tests/generic/group | 1 +
> 3 files changed, 62 insertions(+)
> create mode 100755 tests/generic/586
> create mode 100644 tests/generic/586.out
>
> diff --git a/tests/generic/586 b/tests/generic/586
> new file mode 100755
> index 00000000..5bcad68b
> --- /dev/null
> +++ b/tests/generic/586
> @@ -0,0 +1,59 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Facebook. All Rights Reserved.
> +#
> +# FS QA Test 586
> +#
> +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> +# forever. On other filesystems, this just tests writing into big fallocates.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +. ./common/rc
> +. ./common/filter
> +
> +rm -f $seqres.full
> +
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +
> +maxextlen=$((0x1fffff))
/me wonders if we ought to move this to common/xfs and hoist the other
place (xfs/507) where we define this.
> +bs=4096
> +rextsize=4
> +
> +extra_options=""
> +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> + extra_options="$extra_options -bsize=$bs"
Hm. Could you rework this to _require_realtime, and if the caller
didn't set SCRATCH_RTDEV, create a file on the test device so that we
can always do the realtime test if the kernel supports it? That will
ensure that this gets more testing that it does now...
> + extra_options="$extra_options -r extsize=$((bs * rextsize))"
> + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
...particularly because I don't think very many people actually run
fstests with rt enabled /and/ rtinherit set to stress the realtime
allocator.
(I did a year ago and out came a torrent of bugs such that someone could
probably write themselves a nice year end bonus just fixing all that,
software is terrible :()
> +fi
> +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> +_scratch_mount
> +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> +
> +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> +sync
$XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file
(Hm ok, fallocate the first 2097148 blocks of the file...)
> +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> +sync
$XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file
(now fallocate blocks 2097148 to 2097152)
(Not sure why you do the last rtext separately...)
> +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> +sync
$XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file
(and finally write to block 2097149?)
--D
> +rm "$SCRATCH_MNT/file"
> +
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/generic/586.out b/tests/generic/586.out
> new file mode 100644
> index 00000000..3d36442d
> --- /dev/null
> +++ b/tests/generic/586.out
> @@ -0,0 +1,2 @@
> +QA output created by 586
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index e5d0c1da..6ddaf640 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -588,3 +588,4 @@
> 583 auto quick encrypt
> 584 auto quick encrypt
> 585 auto rename
> +586 auto prealloc preallocrw dangerous
> --
> 2.24.0
>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
2019-11-27 1:26 ` Darrick J. Wong
@ 2019-12-02 19:24 ` Omar Sandoval
2019-12-02 19:31 ` Darrick J. Wong
0 siblings, 1 reply; 4+ messages in thread
From: Omar Sandoval @ 2019-12-02 19:24 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: fstests, kernel-team, linux-xfs
On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote:
> On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> > From: Omar Sandoval <osandov@fb.com>
> >
> > The only XFS-specific part of this test is the setup, so we can make the
> > rest a generic test. It's slow, though, as it needs to write 8GB to
> > convert a big unwritten extent to written.
> >
> > Signed-off-by: Omar Sandoval <osandov@fb.com>
> > ---
> > tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++
> > tests/generic/586.out | 2 ++
> > tests/generic/group | 1 +
> > 3 files changed, 62 insertions(+)
> > create mode 100755 tests/generic/586
> > create mode 100644 tests/generic/586.out
> >
> > diff --git a/tests/generic/586 b/tests/generic/586
> > new file mode 100755
> > index 00000000..5bcad68b
> > --- /dev/null
> > +++ b/tests/generic/586
> > @@ -0,0 +1,59 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2019 Facebook. All Rights Reserved.
> > +#
> > +# FS QA Test 586
> > +#
> > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> > +# forever. On other filesystems, this just tests writing into big fallocates.
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +rm -f $seqres.full
> > +
> > +_supported_fs generic
> > +_supported_os Linux
> > +_require_scratch
> > +
> > +maxextlen=$((0x1fffff))
>
> /me wonders if we ought to move this to common/xfs and hoist the other
> place (xfs/507) where we define this.
>
> > +bs=4096
> > +rextsize=4
> > +
> > +extra_options=""
> > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> > + extra_options="$extra_options -bsize=$bs"
>
> Hm. Could you rework this to _require_realtime, and if the caller
> didn't set SCRATCH_RTDEV, create a file on the test device so that we
> can always do the realtime test if the kernel supports it? That will
> ensure that this gets more testing that it does now...
Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was
set. Did you mean something like this?
if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
use the configured rtdev
else
_require_test
_require_scratch
set up a loop device on the test filesystem as the rtdev
fi
> > + extra_options="$extra_options -r extsize=$((bs * rextsize))"
> > + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
>
> ...particularly because I don't think very many people actually run
> fstests with rt enabled /and/ rtinherit set to stress the realtime
> allocator.
>
> (I did a year ago and out came a torrent of bugs such that someone could
> probably write themselves a nice year end bonus just fixing all that,
> software is terrible :()
>
> > +fi
> > +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> > +_scratch_mount
> > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> > +
> > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> > +sync
>
> $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file
Will fix.
> (Hm ok, fallocate the first 2097148 blocks of the file...)
Oops, I'll add some explanations for all of this stuff.
Thanks for taking a look!
> > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> > +sync
>
> $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file
>
> (now fallocate blocks 2097148 to 2097152)
>
> (Not sure why you do the last rtext separately...)
>
> > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> > +sync
>
> $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file
>
> (and finally write to block 2097149?)
>
> --D
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
2019-12-02 19:24 ` Omar Sandoval
@ 2019-12-02 19:31 ` Darrick J. Wong
0 siblings, 0 replies; 4+ messages in thread
From: Darrick J. Wong @ 2019-12-02 19:31 UTC (permalink / raw)
To: Omar Sandoval; +Cc: fstests, kernel-team, linux-xfs
On Mon, Dec 02, 2019 at 11:24:24AM -0800, Omar Sandoval wrote:
> On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote:
> > On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> > > From: Omar Sandoval <osandov@fb.com>
> > >
> > > The only XFS-specific part of this test is the setup, so we can make the
> > > rest a generic test. It's slow, though, as it needs to write 8GB to
> > > convert a big unwritten extent to written.
> > >
> > > Signed-off-by: Omar Sandoval <osandov@fb.com>
> > > ---
> > > tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++
> > > tests/generic/586.out | 2 ++
> > > tests/generic/group | 1 +
> > > 3 files changed, 62 insertions(+)
> > > create mode 100755 tests/generic/586
> > > create mode 100644 tests/generic/586.out
> > >
> > > diff --git a/tests/generic/586 b/tests/generic/586
> > > new file mode 100755
> > > index 00000000..5bcad68b
> > > --- /dev/null
> > > +++ b/tests/generic/586
> > > @@ -0,0 +1,59 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2019 Facebook. All Rights Reserved.
> > > +#
> > > +# FS QA Test 586
> > > +#
> > > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> > > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> > > +# forever. On other filesystems, this just tests writing into big fallocates.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +. ./common/rc
> > > +. ./common/filter
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_supported_fs generic
> > > +_supported_os Linux
> > > +_require_scratch
> > > +
> > > +maxextlen=$((0x1fffff))
> >
> > /me wonders if we ought to move this to common/xfs and hoist the other
> > place (xfs/507) where we define this.
> >
> > > +bs=4096
> > > +rextsize=4
> > > +
> > > +extra_options=""
> > > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> > > + extra_options="$extra_options -bsize=$bs"
> >
> > Hm. Could you rework this to _require_realtime, and if the caller
> > didn't set SCRATCH_RTDEV, create a file on the test device so that we
> > can always do the realtime test if the kernel supports it? That will
> > ensure that this gets more testing that it does now...
>
> Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was
> set. Did you mean something like this?
>
> if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> use the configured rtdev
> else
> _require_test
> _require_scratch
> set up a loop device on the test filesystem as the rtdev
> fi
Yep, that's exactly what I meant.
--D
> > > + extra_options="$extra_options -r extsize=$((bs * rextsize))"
> > > + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
> >
> > ...particularly because I don't think very many people actually run
> > fstests with rt enabled /and/ rtinherit set to stress the realtime
> > allocator.
> >
> > (I did a year ago and out came a torrent of bugs such that someone could
> > probably write themselves a nice year end bonus just fixing all that,
> > software is terrible :()
> >
> > > +fi
> > > +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> > > +
> > > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> > > +sync
> >
> > $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file
>
> Will fix.
>
> > (Hm ok, fallocate the first 2097148 blocks of the file...)
>
> Oops, I'll add some explanations for all of this stuff.
>
> Thanks for taking a look!
>
> > > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> > > +sync
> >
> > $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file
> >
> > (now fallocate blocks 2097148 to 2097152)
> >
> > (Not sure why you do the last rtext separately...)
> >
> > > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> > > +sync
> >
> > $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file
> >
> > (and finally write to block 2097149?)
> >
> > --D
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-12-02 19:31 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-26 20:13 [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent Omar Sandoval
2019-11-27 1:26 ` Darrick J. Wong
2019-12-02 19:24 ` Omar Sandoval
2019-12-02 19:31 ` Darrick J. Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).