* [PATCH] generic: shutdown might leave NULL files with nonzero di_size
@ 2022-11-04 16:20 Zorro Lang
2022-11-04 16:29 ` Darrick J. Wong
0 siblings, 1 reply; 4+ messages in thread
From: Zorro Lang @ 2022-11-04 16:20 UTC (permalink / raw)
To: fstests; +Cc: linux-xfs
An old issue might cause on-disk inode sizes are logged prematurely
via the free eofblocks path on file close. Then fs shutdown might
leave NULL files but their di_size > 0.
Signed-off-by: Zorro Lang <zlang@kernel.org>
---
Hi,
There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to
fstests. I've tried generic/044~049, no one can reproduce this bug, so I
have to write this new one. It fails on rhel-6.5 [1], and test passed on
later kernel.
I hard to say which patch fix this issue exactly, it's fixed by a patchset
which does code improvement/cleanup.
Thanks,
Zorro
[1]
# ./check generic/999
FSTYP -- xfs (non-debug)
PLATFORM -- Linux/x86_64
MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1
MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratch
generic/999 2s ... - output mismatch (see /root/xfstests-dev/results//generic/999.out.bad)
--- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400
+++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400
@@ -1 +1,3 @@
QA output created by 999
+ - /mnt/scratch/1 get no extents, but its di_size > 0
+/mnt/scratch/1:
...
(Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)
Ran: generic/999
Failures: generic/999
Failed 1 of 1 tests
tests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/999.out | 5 +++++
2 files changed, 51 insertions(+)
create mode 100755 tests/generic/999
create mode 100644 tests/generic/999.out
diff --git a/tests/generic/999 b/tests/generic/999
new file mode 100755
index 00000000..a2e662fc
--- /dev/null
+++ b/tests/generic/999
@@ -0,0 +1,46 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved.
+#
+# FS QA Test No. 999
+#
+# Test an issue in the truncate codepath where on-disk inode sizes are logged
+# prematurely via the free eofblocks path on file close.
+#
+. ./common/preamble
+_begin_fstest auto quick shutdown
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_xfs_io_command fiemap
+_require_scratch_shutdown
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount
+
+echo "Create many small files with one extent at least"
+for ((i=0; i<10000; i++));do
+ $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1
+done
+
+echo "Shutdown the fs suddently"
+_scratch_shutdown
+
+echo "Cycle mount"
+_scratch_cycle_mount
+
+echo "Check file's (di_size > 0) extents"
+for f in $(find $SCRATCH_MNT -type f -size +0);do
+ $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap
+ # Check if the file has any extent
+ grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap
+ if [ $? -ne 0 ];then
+ echo " - $f get no extents, but its di_size > 0"
+ cat $tmp.fiemap
+ break
+ fi
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/999.out b/tests/generic/999.out
new file mode 100644
index 00000000..50008783
--- /dev/null
+++ b/tests/generic/999.out
@@ -0,0 +1,5 @@
+QA output created by 999
+Create many small files with one extent at least
+Shutdown the fs suddently
+Cycle mount
+Check file's (di_size > 0) extents
--
2.31.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] generic: shutdown might leave NULL files with nonzero di_size
2022-11-04 16:20 [PATCH] generic: shutdown might leave NULL files with nonzero di_size Zorro Lang
@ 2022-11-04 16:29 ` Darrick J. Wong
2022-11-05 12:16 ` Zorro Lang
0 siblings, 1 reply; 4+ messages in thread
From: Darrick J. Wong @ 2022-11-04 16:29 UTC (permalink / raw)
To: Zorro Lang; +Cc: fstests, linux-xfs
On Sat, Nov 05, 2022 at 12:20:02AM +0800, Zorro Lang wrote:
> An old issue might cause on-disk inode sizes are logged prematurely
> via the free eofblocks path on file close. Then fs shutdown might
> leave NULL files but their di_size > 0.
>
> Signed-off-by: Zorro Lang <zlang@kernel.org>
> ---
>
> Hi,
>
> There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to
> fstests. I've tried generic/044~049, no one can reproduce this bug, so I
> have to write this new one. It fails on rhel-6.5 [1], and test passed on
> later kernel.
>
> I hard to say which patch fix this issue exactly, it's fixed by a patchset
> which does code improvement/cleanup.
>
> Thanks,
> Zorro
>
> [1]
> # ./check generic/999
> FSTYP -- xfs (non-debug)
> PLATFORM -- Linux/x86_64
> MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1
> MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratch
>
> generic/999 2s ... - output mismatch (see /root/xfstests-dev/results//generic/999.out.bad)
> --- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400
> +++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400
> @@ -1 +1,3 @@
> QA output created by 999
> + - /mnt/scratch/1 get no extents, but its di_size > 0
> +/mnt/scratch/1:
> ...
> (Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)
> Ran: generic/999
> Failures: generic/999
> Failed 1 of 1 tests
>
> tests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/999.out | 5 +++++
> 2 files changed, 51 insertions(+)
> create mode 100755 tests/generic/999
> create mode 100644 tests/generic/999.out
>
> diff --git a/tests/generic/999 b/tests/generic/999
> new file mode 100755
> index 00000000..a2e662fc
> --- /dev/null
> +++ b/tests/generic/999
> @@ -0,0 +1,46 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved.
> +#
> +# FS QA Test No. 999
> +#
> +# Test an issue in the truncate codepath where on-disk inode sizes are logged
> +# prematurely via the free eofblocks path on file close.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick shutdown
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_scratch
> +_require_xfs_io_command fiemap
/me would've thought you'd use the xfs_io stat/bmap commands to detect
either nextents > 0 (stat) or actual mappings returned (bmap), but I
guess if RHEL 6.5 xfsprogs has a fiemap command then this is fine with
me.
If the answer to the above is "um, RHEL 6.5 xfsprogs *does* have FIEMAP",
then there's little point in rewriting a stable regression test, so:
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
--D
> +_require_scratch_shutdown
> +_scratch_mkfs > $seqres.full 2>&1
> +_scratch_mount
> +
> +echo "Create many small files with one extent at least"
> +for ((i=0; i<10000; i++));do
> + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1
> +done
> +
> +echo "Shutdown the fs suddently"
> +_scratch_shutdown
> +
> +echo "Cycle mount"
> +_scratch_cycle_mount
> +
> +echo "Check file's (di_size > 0) extents"
> +for f in $(find $SCRATCH_MNT -type f -size +0);do
> + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap
> + # Check if the file has any extent
> + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap
> + if [ $? -ne 0 ];then
> + echo " - $f get no extents, but its di_size > 0"
> + cat $tmp.fiemap
> + break
> + fi
> +done
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/999.out b/tests/generic/999.out
> new file mode 100644
> index 00000000..50008783
> --- /dev/null
> +++ b/tests/generic/999.out
> @@ -0,0 +1,5 @@
> +QA output created by 999
> +Create many small files with one extent at least
> +Shutdown the fs suddently
> +Cycle mount
> +Check file's (di_size > 0) extents
> --
> 2.31.1
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] generic: shutdown might leave NULL files with nonzero di_size
2022-11-04 16:29 ` Darrick J. Wong
@ 2022-11-05 12:16 ` Zorro Lang
2022-11-05 12:28 ` Zorro Lang
0 siblings, 1 reply; 4+ messages in thread
From: Zorro Lang @ 2022-11-05 12:16 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Zorro Lang, fstests, linux-xfs
On Fri, Nov 04, 2022 at 09:29:28AM -0700, Darrick J. Wong wrote:
> On Sat, Nov 05, 2022 at 12:20:02AM +0800, Zorro Lang wrote:
> > An old issue might cause on-disk inode sizes are logged prematurely
> > via the free eofblocks path on file close. Then fs shutdown might
> > leave NULL files but their di_size > 0.
> >
> > Signed-off-by: Zorro Lang <zlang@kernel.org>
> > ---
> >
> > Hi,
> >
> > There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to
> > fstests. I've tried generic/044~049, no one can reproduce this bug, so I
> > have to write this new one. It fails on rhel-6.5 [1], and test passed on
> > later kernel.
> >
> > I hard to say which patch fix this issue exactly, it's fixed by a patchset
> > which does code improvement/cleanup.
> >
> > Thanks,
> > Zorro
> >
> > [1]
> > # ./check generic/999
> > FSTYP -- xfs (non-debug)
> > PLATFORM -- Linux/x86_64
> > MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1
> > MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratch
> >
> > generic/999 2s ... - output mismatch (see /root/xfstests-dev/results//generic/999.out.bad)
> > --- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400
> > +++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400
> > @@ -1 +1,3 @@
> > QA output created by 999
> > + - /mnt/scratch/1 get no extents, but its di_size > 0
> > +/mnt/scratch/1:
> > ...
> > (Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)
> > Ran: generic/999
> > Failures: generic/999
> > Failed 1 of 1 tests
> >
> > tests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++
> > tests/generic/999.out | 5 +++++
> > 2 files changed, 51 insertions(+)
> > create mode 100755 tests/generic/999
> > create mode 100644 tests/generic/999.out
> >
> > diff --git a/tests/generic/999 b/tests/generic/999
> > new file mode 100755
> > index 00000000..a2e662fc
> > --- /dev/null
> > +++ b/tests/generic/999
> > @@ -0,0 +1,46 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test No. 999
> > +#
> > +# Test an issue in the truncate codepath where on-disk inode sizes are logged
> > +# prematurely via the free eofblocks path on file close.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick shutdown
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_require_scratch
> > +_require_xfs_io_command fiemap
>
> /me would've thought you'd use the xfs_io stat/bmap commands to detect
> either nextents > 0 (stat) or actual mappings returned (bmap), but I
> guess if RHEL 6.5 xfsprogs has a fiemap command then this is fine with
> me.
Ah, you're right, rhel-6 xfs_io doesn't support fiemap :)
And yes, I wrote this case as a xfs specific case at first, by using xfs_bmap.
The original case (of us) uses xfs_bmap too. But I thought fiemap can help it
to be a generic case to cover more fs, so I turn to use fiemap.
>
> If the answer to the above is "um, RHEL 6.5 xfsprogs *does* have FIEMAP",
> then there's little point in rewriting a stable regression test, so:
I don't worry about rhel6 testing, due to I keep our internal case (for rhel6
only). I'd like to see more generic cases, except fiemap can't help this test :)
Thanks,
Zorro
>
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>
> --D
>
> > +_require_scratch_shutdown
> > +_scratch_mkfs > $seqres.full 2>&1
> > +_scratch_mount
> > +
> > +echo "Create many small files with one extent at least"
> > +for ((i=0; i<10000; i++));do
> > + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1
> > +done
> > +
> > +echo "Shutdown the fs suddently"
> > +_scratch_shutdown
> > +
> > +echo "Cycle mount"
> > +_scratch_cycle_mount
> > +
> > +echo "Check file's (di_size > 0) extents"
> > +for f in $(find $SCRATCH_MNT -type f -size +0);do
> > + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap
> > + # Check if the file has any extent
> > + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap
> > + if [ $? -ne 0 ];then
> > + echo " - $f get no extents, but its di_size > 0"
> > + cat $tmp.fiemap
> > + break
> > + fi
> > +done
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/999.out b/tests/generic/999.out
> > new file mode 100644
> > index 00000000..50008783
> > --- /dev/null
> > +++ b/tests/generic/999.out
> > @@ -0,0 +1,5 @@
> > +QA output created by 999
> > +Create many small files with one extent at least
> > +Shutdown the fs suddently
> > +Cycle mount
> > +Check file's (di_size > 0) extents
> > --
> > 2.31.1
> >
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] generic: shutdown might leave NULL files with nonzero di_size
2022-11-05 12:16 ` Zorro Lang
@ 2022-11-05 12:28 ` Zorro Lang
0 siblings, 0 replies; 4+ messages in thread
From: Zorro Lang @ 2022-11-05 12:28 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Zorro Lang, fstests, linux-xfs
On Sat, Nov 05, 2022 at 08:16:11PM +0800, Zorro Lang wrote:
> On Fri, Nov 04, 2022 at 09:29:28AM -0700, Darrick J. Wong wrote:
> > On Sat, Nov 05, 2022 at 12:20:02AM +0800, Zorro Lang wrote:
> > > An old issue might cause on-disk inode sizes are logged prematurely
> > > via the free eofblocks path on file close. Then fs shutdown might
> > > leave NULL files but their di_size > 0.
> > >
> > > Signed-off-by: Zorro Lang <zlang@kernel.org>
> > > ---
> > >
> > > Hi,
> > >
> > > There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to
> > > fstests. I've tried generic/044~049, no one can reproduce this bug, so I
> > > have to write this new one. It fails on rhel-6.5 [1], and test passed on
> > > later kernel.
> > >
> > > I hard to say which patch fix this issue exactly, it's fixed by a patchset
> > > which does code improvement/cleanup.
> > >
> > > Thanks,
> > > Zorro
> > >
> > > [1]
> > > # ./check generic/999
> > > FSTYP -- xfs (non-debug)
> > > PLATFORM -- Linux/x86_64
> > > MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1
> > > MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratch
> > >
> > > generic/999 2s ... - output mismatch (see /root/xfstests-dev/results//generic/999.out.bad)
> > > --- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400
> > > +++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400
> > > @@ -1 +1,3 @@
> > > QA output created by 999
> > > + - /mnt/scratch/1 get no extents, but its di_size > 0
> > > +/mnt/scratch/1:
> > > ...
> > > (Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)
> > > Ran: generic/999
> > > Failures: generic/999
> > > Failed 1 of 1 tests
> > >
> > > tests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++
> > > tests/generic/999.out | 5 +++++
> > > 2 files changed, 51 insertions(+)
> > > create mode 100755 tests/generic/999
> > > create mode 100644 tests/generic/999.out
> > >
> > > diff --git a/tests/generic/999 b/tests/generic/999
> > > new file mode 100755
> > > index 00000000..a2e662fc
> > > --- /dev/null
> > > +++ b/tests/generic/999
> > > @@ -0,0 +1,46 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved.
> > > +#
> > > +# FS QA Test No. 999
> > > +#
> > > +# Test an issue in the truncate codepath where on-disk inode sizes are logged
> > > +# prematurely via the free eofblocks path on file close.
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest auto quick shutdown
> > > +
> > > +# real QA test starts here
> > > +_supported_fs xfs
> > > +_require_scratch
> > > +_require_xfs_io_command fiemap
> >
> > /me would've thought you'd use the xfs_io stat/bmap commands to detect
> > either nextents > 0 (stat) or actual mappings returned (bmap), but I
Wait a minute... the "stat" command can help this case too?
I didn't use bmap command due to it only for xfs, but stat is a common command,
if it helps, that's better than fiemap (due to it supports old system). I'll
give it a test, and will resend the patch if it works. Thanks your suggestion!
> > guess if RHEL 6.5 xfsprogs has a fiemap command then this is fine with
> > me.
>
> Ah, you're right, rhel-6 xfs_io doesn't support fiemap :)
>
> And yes, I wrote this case as a xfs specific case at first, by using xfs_bmap.
> The original case (of us) uses xfs_bmap too. But I thought fiemap can help it
> to be a generic case to cover more fs, so I turn to use fiemap.
>
> >
> > If the answer to the above is "um, RHEL 6.5 xfsprogs *does* have FIEMAP",
> > then there's little point in rewriting a stable regression test, so:
>
> I don't worry about rhel6 testing, due to I keep our internal case (for rhel6
> only). I'd like to see more generic cases, except fiemap can't help this test :)
>
> Thanks,
> Zorro
>
> >
> > Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> >
> > --D
> >
> > > +_require_scratch_shutdown
> > > +_scratch_mkfs > $seqres.full 2>&1
> > > +_scratch_mount
> > > +
> > > +echo "Create many small files with one extent at least"
> > > +for ((i=0; i<10000; i++));do
> > > + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1
> > > +done
> > > +
> > > +echo "Shutdown the fs suddently"
> > > +_scratch_shutdown
> > > +
> > > +echo "Cycle mount"
> > > +_scratch_cycle_mount
> > > +
> > > +echo "Check file's (di_size > 0) extents"
> > > +for f in $(find $SCRATCH_MNT -type f -size +0);do
> > > + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap
> > > + # Check if the file has any extent
> > > + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap
> > > + if [ $? -ne 0 ];then
> > > + echo " - $f get no extents, but its di_size > 0"
> > > + cat $tmp.fiemap
> > > + break
> > > + fi
> > > +done
> > > +
> > > +# success, all done
> > > +status=0
> > > +exit
> > > diff --git a/tests/generic/999.out b/tests/generic/999.out
> > > new file mode 100644
> > > index 00000000..50008783
> > > --- /dev/null
> > > +++ b/tests/generic/999.out
> > > @@ -0,0 +1,5 @@
> > > +QA output created by 999
> > > +Create many small files with one extent at least
> > > +Shutdown the fs suddently
> > > +Cycle mount
> > > +Check file's (di_size > 0) extents
> > > --
> > > 2.31.1
> > >
> >
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-11-05 12:30 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-04 16:20 [PATCH] generic: shutdown might leave NULL files with nonzero di_size Zorro Lang
2022-11-04 16:29 ` Darrick J. Wong
2022-11-05 12:16 ` Zorro Lang
2022-11-05 12:28 ` Zorro Lang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.