Linux-XFS Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
@ 2019-11-26 20:13 Omar Sandoval
  2019-11-27  1:26 ` Darrick J. Wong
  0 siblings, 1 reply; 4+ messages in thread
From: Omar Sandoval @ 2019-11-26 20:13 UTC (permalink / raw)
  To: fstests; +Cc: kernel-team, linux-xfs

From: Omar Sandoval <osandov@fb.com>

The only XFS-specific part of this test is the setup, so we can make the
rest a generic test. It's slow, though, as it needs to write 8GB to
convert a big unwritten extent to written.

Signed-off-by: Omar Sandoval <osandov@fb.com>
---
 tests/generic/586     | 59 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/586.out |  2 ++
 tests/generic/group   |  1 +
 3 files changed, 62 insertions(+)
 create mode 100755 tests/generic/586
 create mode 100644 tests/generic/586.out

diff --git a/tests/generic/586 b/tests/generic/586
new file mode 100755
index 00000000..5bcad68b
--- /dev/null
+++ b/tests/generic/586
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Facebook.  All Rights Reserved.
+#
+# FS QA Test 586
+#
+# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
+# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
+# forever. On other filesystems, this just tests writing into big fallocates.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+. ./common/rc
+. ./common/filter
+
+rm -f $seqres.full
+
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+maxextlen=$((0x1fffff))
+bs=4096
+rextsize=4
+
+extra_options=""
+if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
+	extra_options="$extra_options -bsize=$bs"
+	extra_options="$extra_options -r extsize=$((bs * rextsize))"
+	extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
+fi
+_scratch_mkfs $extra_options >>$seqres.full 2>&1
+_scratch_mount
+_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
+
+fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
+sync
+fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
+sync
+dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
+sync
+rm "$SCRATCH_MNT/file"
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/generic/586.out b/tests/generic/586.out
new file mode 100644
index 00000000..3d36442d
--- /dev/null
+++ b/tests/generic/586.out
@@ -0,0 +1,2 @@
+QA output created by 586
+Silence is golden
diff --git a/tests/generic/group b/tests/generic/group
index e5d0c1da..6ddaf640 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -588,3 +588,4 @@
 583 auto quick encrypt
 584 auto quick encrypt
 585 auto rename
+586 auto prealloc preallocrw dangerous
-- 
2.24.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
  2019-11-26 20:13 [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent Omar Sandoval
@ 2019-11-27  1:26 ` Darrick J. Wong
  2019-12-02 19:24   ` Omar Sandoval
  0 siblings, 1 reply; 4+ messages in thread
From: Darrick J. Wong @ 2019-11-27  1:26 UTC (permalink / raw)
  To: Omar Sandoval; +Cc: fstests, kernel-team, linux-xfs

On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
> 
> The only XFS-specific part of this test is the setup, so we can make the
> rest a generic test. It's slow, though, as it needs to write 8GB to
> convert a big unwritten extent to written.
> 
> Signed-off-by: Omar Sandoval <osandov@fb.com>
> ---
>  tests/generic/586     | 59 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/586.out |  2 ++
>  tests/generic/group   |  1 +
>  3 files changed, 62 insertions(+)
>  create mode 100755 tests/generic/586
>  create mode 100644 tests/generic/586.out
> 
> diff --git a/tests/generic/586 b/tests/generic/586
> new file mode 100755
> index 00000000..5bcad68b
> --- /dev/null
> +++ b/tests/generic/586
> @@ -0,0 +1,59 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2019 Facebook.  All Rights Reserved.
> +#
> +# FS QA Test 586
> +#
> +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> +# forever. On other filesystems, this just tests writing into big fallocates.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +. ./common/rc
> +. ./common/filter
> +
> +rm -f $seqres.full
> +
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +
> +maxextlen=$((0x1fffff))

/me wonders if we ought to move this to common/xfs and hoist the other
place (xfs/507) where we define this.

> +bs=4096
> +rextsize=4
> +
> +extra_options=""
> +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> +	extra_options="$extra_options -bsize=$bs"

Hm.  Could you rework this to _require_realtime, and if the caller
didn't set SCRATCH_RTDEV, create a file on the test device so that we
can always do the realtime test if the kernel supports it?  That will
ensure that this gets more testing that it does now...

> +	extra_options="$extra_options -r extsize=$((bs * rextsize))"
> +	extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"

...particularly because I don't think very many people actually run
fstests with rt enabled /and/ rtinherit set to stress the realtime
allocator.

(I did a year ago and out came a torrent of bugs such that someone could
probably write themselves a nice year end bonus just fixing all that,
software is terrible :()

> +fi
> +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> +_scratch_mount
> +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> +
> +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> +sync

$XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file

(Hm ok, fallocate the first 2097148 blocks of the file...)

> +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> +sync

$XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file

(now fallocate blocks 2097148 to 2097152)

(Not sure why you do the last rtext separately...)

> +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> +sync

$XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file

(and finally write to block 2097149?)

--D

> +rm "$SCRATCH_MNT/file"
> +
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/generic/586.out b/tests/generic/586.out
> new file mode 100644
> index 00000000..3d36442d
> --- /dev/null
> +++ b/tests/generic/586.out
> @@ -0,0 +1,2 @@
> +QA output created by 586
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index e5d0c1da..6ddaf640 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -588,3 +588,4 @@
>  583 auto quick encrypt
>  584 auto quick encrypt
>  585 auto rename
> +586 auto prealloc preallocrw dangerous
> -- 
> 2.24.0
> 

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
  2019-11-27  1:26 ` Darrick J. Wong
@ 2019-12-02 19:24   ` Omar Sandoval
  2019-12-02 19:31     ` Darrick J. Wong
  0 siblings, 1 reply; 4+ messages in thread
From: Omar Sandoval @ 2019-12-02 19:24 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: fstests, kernel-team, linux-xfs

On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote:
> On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> > From: Omar Sandoval <osandov@fb.com>
> > 
> > The only XFS-specific part of this test is the setup, so we can make the
> > rest a generic test. It's slow, though, as it needs to write 8GB to
> > convert a big unwritten extent to written.
> > 
> > Signed-off-by: Omar Sandoval <osandov@fb.com>
> > ---
> >  tests/generic/586     | 59 +++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/586.out |  2 ++
> >  tests/generic/group   |  1 +
> >  3 files changed, 62 insertions(+)
> >  create mode 100755 tests/generic/586
> >  create mode 100644 tests/generic/586.out
> > 
> > diff --git a/tests/generic/586 b/tests/generic/586
> > new file mode 100755
> > index 00000000..5bcad68b
> > --- /dev/null
> > +++ b/tests/generic/586
> > @@ -0,0 +1,59 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2019 Facebook.  All Rights Reserved.
> > +#
> > +# FS QA Test 586
> > +#
> > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> > +# forever. On other filesystems, this just tests writing into big fallocates.
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1	# failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +rm -f $seqres.full
> > +
> > +_supported_fs generic
> > +_supported_os Linux
> > +_require_scratch
> > +
> > +maxextlen=$((0x1fffff))
> 
> /me wonders if we ought to move this to common/xfs and hoist the other
> place (xfs/507) where we define this.
> 
> > +bs=4096
> > +rextsize=4
> > +
> > +extra_options=""
> > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> > +	extra_options="$extra_options -bsize=$bs"
> 
> Hm.  Could you rework this to _require_realtime, and if the caller
> didn't set SCRATCH_RTDEV, create a file on the test device so that we
> can always do the realtime test if the kernel supports it?  That will
> ensure that this gets more testing that it does now...

Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was
set. Did you mean something like this?

if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
	use the configured rtdev
else
	_require_test
	_require_scratch
	set up a loop device on the test filesystem as the rtdev
fi

> > +	extra_options="$extra_options -r extsize=$((bs * rextsize))"
> > +	extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
> 
> ...particularly because I don't think very many people actually run
> fstests with rt enabled /and/ rtinherit set to stress the realtime
> allocator.
> 
> (I did a year ago and out came a torrent of bugs such that someone could
> probably write themselves a nice year end bonus just fixing all that,
> software is terrible :()
> 
> > +fi
> > +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> > +_scratch_mount
> > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> > +
> > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> > +sync
> 
> $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file

Will fix.

> (Hm ok, fallocate the first 2097148 blocks of the file...)

Oops, I'll add some explanations for all of this stuff.

Thanks for taking a look!

> > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> > +sync
> 
> $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file
> 
> (now fallocate blocks 2097148 to 2097152)
> 
> (Not sure why you do the last rtext separately...)
> 
> > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> > +sync
> 
> $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file
> 
> (and finally write to block 2097149?)
> 
> --D

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent
  2019-12-02 19:24   ` Omar Sandoval
@ 2019-12-02 19:31     ` Darrick J. Wong
  0 siblings, 0 replies; 4+ messages in thread
From: Darrick J. Wong @ 2019-12-02 19:31 UTC (permalink / raw)
  To: Omar Sandoval; +Cc: fstests, kernel-team, linux-xfs

On Mon, Dec 02, 2019 at 11:24:24AM -0800, Omar Sandoval wrote:
> On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote:
> > On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote:
> > > From: Omar Sandoval <osandov@fb.com>
> > > 
> > > The only XFS-specific part of this test is the setup, so we can make the
> > > rest a generic test. It's slow, though, as it needs to write 8GB to
> > > convert a big unwritten extent to written.
> > > 
> > > Signed-off-by: Omar Sandoval <osandov@fb.com>
> > > ---
> > >  tests/generic/586     | 59 +++++++++++++++++++++++++++++++++++++++++++
> > >  tests/generic/586.out |  2 ++
> > >  tests/generic/group   |  1 +
> > >  3 files changed, 62 insertions(+)
> > >  create mode 100755 tests/generic/586
> > >  create mode 100644 tests/generic/586.out
> > > 
> > > diff --git a/tests/generic/586 b/tests/generic/586
> > > new file mode 100755
> > > index 00000000..5bcad68b
> > > --- /dev/null
> > > +++ b/tests/generic/586
> > > @@ -0,0 +1,59 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2019 Facebook.  All Rights Reserved.
> > > +#
> > > +# FS QA Test 586
> > > +#
> > > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG
> > > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang
> > > +# forever. On other filesystems, this just tests writing into big fallocates.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1	# failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > +	cd /
> > > +	rm -f $tmp.*
> > > +}
> > > +
> > > +. ./common/rc
> > > +. ./common/filter
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_supported_fs generic
> > > +_supported_os Linux
> > > +_require_scratch
> > > +
> > > +maxextlen=$((0x1fffff))
> > 
> > /me wonders if we ought to move this to common/xfs and hoist the other
> > place (xfs/507) where we define this.
> > 
> > > +bs=4096
> > > +rextsize=4
> > > +
> > > +extra_options=""
> > > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> > > +	extra_options="$extra_options -bsize=$bs"
> > 
> > Hm.  Could you rework this to _require_realtime, and if the caller
> > didn't set SCRATCH_RTDEV, create a file on the test device so that we
> > can always do the realtime test if the kernel supports it?  That will
> > ensure that this gets more testing that it does now...
> 
> Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was
> set. Did you mean something like this?
> 
> if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then
> 	use the configured rtdev
> else
> 	_require_test
> 	_require_scratch
> 	set up a loop device on the test filesystem as the rtdev
> fi

Yep, that's exactly what I meant.

--D

> > > +	extra_options="$extra_options -r extsize=$((bs * rextsize))"
> > > +	extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1"
> > 
> > ...particularly because I don't think very many people actually run
> > fstests with rt enabled /and/ rtinherit set to stress the realtime
> > allocator.
> > 
> > (I did a year ago and out came a torrent of bugs such that someone could
> > probably write themselves a nice year end bonus just fixing all that,
> > software is terrible :()
> > 
> > > +fi
> > > +_scratch_mkfs $extra_options >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))"
> > > +
> > > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file"
> > > +sync
> > 
> > $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file
> 
> Will fix.
> 
> > (Hm ok, fallocate the first 2097148 blocks of the file...)
> 
> Oops, I'll add some explanations for all of this stuff.
> 
> Thanks for taking a look!
> 
> > > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file"
> > > +sync
> > 
> > $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file
> > 
> > (now fallocate blocks 2097148 to 2097152)
> > 
> > (Not sure why you do the last rtext separately...)
> > 
> > > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none
> > > +sync
> > 
> > $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file
> > 
> > (and finally write to block 2097149?)
> > 
> > --D

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, back to index

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-26 20:13 [PATCH] generic: test truncating mixed written/unwritten XFS realtime extent Omar Sandoval
2019-11-27  1:26 ` Darrick J. Wong
2019-12-02 19:24   ` Omar Sandoval
2019-12-02 19:31     ` Darrick J. Wong

Linux-XFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-xfs/0 linux-xfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-xfs linux-xfs/ https://lore.kernel.org/linux-xfs \
		linux-xfs@vger.kernel.org
	public-inbox-index linux-xfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-xfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git