From: "Darrick J. Wong" <djwong@kernel.org>
To: Chandan Babu R <chandanrlinux@gmail.com>
Cc: fstests@vger.kernel.org, linux-xfs@vger.kernel.org
Subject: Re: [PATCH V6 05/13] xfs: Check for extent overflow when growing realtime bitmap/summary inodes
Date: Mon, 22 Mar 2021 10:56:52 -0700 [thread overview]
Message-ID: <20210322175652.GG1670408@magnolia> (raw)
In-Reply-To: <20210309050124.23797-6-chandanrlinux@gmail.com>
On Tue, Mar 09, 2021 at 10:31:16AM +0530, Chandan Babu R wrote:
> Verify that XFS does not cause realtime bitmap/summary inode fork's
> extent count to overflow when growing the realtime volume associated
> with a filesystem.
>
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
Soo... I discovered that this test doesn't pass with multiblock
directories:
FSTYP -- xfs (debug)
PLATFORM -- Linux/x86_64 alder-mtr00 5.12.0-rc4-xfsx #rc4 SMP PREEMPT Mon Mar 22 10:03:45 PDT 2021
MKFS_OPTIONS -- -f -b size=1024, /dev/sdf
MOUNT_OPTIONS -- -o usrquota,grpquota,prjquota, /dev/sdf /opt
xfs/529 - output mismatch (see /var/tmp/fstests/xfs/529.out.bad)
--- tests/xfs/529.out 2021-03-21 11:44:09.383407733 -0700
+++ /var/tmp/fstests/xfs/529.out.bad 2021-03-22 10:36:34.000348426 -0700
@@ -4,12 +4,21 @@
Inject reduce_max_iextents error tag
Create fragmented file
Verify $testfile's extent count
+/opt/testfile: No such file or directory
+/tmp/fstests/tests/xfs/529: line 72: ((: > 10 : syntax error: operand expected (error token is "> 10 ")
+rm: cannot remove '/opt/testfile': No such file or directory
* Fallocate unwritten extents
...
(Run 'diff -u /tmp/fstests/tests/xfs/529.out /var/tmp/fstests/xfs/529.out.bad' to see the entire diff)
Ran: xfs/529
Failures: xfs/529
Failed 1 of 1 tests
Test xfs/529 FAILED with code 1 and bad golden output:
--- /tmp/fstests/tests/xfs/529.out 2021-03-21 11:44:09.383407733 -0700
+++ /var/tmp/fstests/xfs/529.out.bad 2021-03-22 10:36:34.000348426 -0700
@@ -4,12 +4,21 @@
Inject reduce_max_iextents error tag
Create fragmented file
Verify $testfile's extent count
+/opt/testfile: No such file or directory
+/tmp/fstests/tests/xfs/529: line 72: ((: > 10 : syntax error: operand expected (error token is "> 10 ")
+rm: cannot remove '/opt/testfile': No such file or directory
* Fallocate unwritten extents
Fallocate fragmented file
Verify $testfile's extent count
+/opt/testfile: No such file or directory
+/tmp/fstests/tests/xfs/529: line 91: ((: > 10 : syntax error: operand expected (error token is "> 10 ")
+rm: cannot remove '/opt/testfile': No such file or directory
* Directio write
Create fragmented file via directio writes
Verify $testfile's extent count
+/opt/testfile: No such file or directory
+/tmp/fstests/tests/xfs/529: line 110: ((: > 10 : syntax error: operand expected (error token is "> 10 ")
+rm: cannot remove '/opt/testfile': No such file or directory
* Extend quota inodes
Disable reduce_max_iextents error tag
Consume free space
The test appears to fail because we cannot create even a single file in
the root directory. Looking at xfs_create, I see:
error = xfs_iext_count_may_overflow(dp, XFS_DATA_FORK,
XFS_IEXT_DIR_MANIP_CNT(mp));
if (error)
goto out_trans_cancel;
XFS_IEXT_DIR_MANIP_CNT is defined as:
#define XFS_IEXT_DIR_MANIP_CNT(mp) \
((XFS_DA_NODE_MAXDEPTH + 1 + 1) * (mp)->m_dir_geo->fsbcount)
If one formats a filesystem with 1k blocks, the result will be a
filesystem with 4k directory blocks:
# mkfs.xfs -b size=1024 /dev/sdf -Nf
meta-data=/dev/sdf isize=512 agcount=4, agsize=5192704 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=1
= reflink=1 bigtime=1 inobtcount=1
= metadir=0
data = bsize=1024 blocks=20770816, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=1024 blocks=10240, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Note "data bsize" is 1024, and "naming bsize" is 4096.
In the kernel, we set m_dir_geo->fsbcount = "naming bsize" /
"data bsize", or 4 in this case. Since XFS_DA_NODE_MAXDEPTH is always
5, this macro expands to:
(5 + 1 + 1) * (4) = 28
The reason for the test failure I think is because of this code in
xfs_iext_count_may_overflow, which is called from xfs_create on the
parent directory:
if (XFS_TEST_ERROR(false, ip->i_mount, XFS_ERRTAG_REDUCE_MAX_IEXTENTS))
max_exts = 10;
nr_exts = ifp->if_nextents + nr_to_add;
if (nr_exts < ifp->if_nextents || nr_exts > max_ext)
return -EFBIG
The second part of the if statement becomes (28 > 10) which is trivially
true, so we return -EFBIG for all attempts to create a file in a
directory. xfs/529, in turn, cannot create $testfile because nothing
can create a file in $SCRATCH_MNT, and the test goes off the rails.
I think this can be trivially solved by changing this (and the other
tests) to ensure that the error injection is only set when we're running
a command to check if we get EFBIG. In other words, this code in
xfs/529:
rm $testfile
echo "* Fallocate unwritten extents"
echo "Fallocate fragmented file"
for i in $(seq 0 2 $((nr_blks - 1))); do
$XFS_IO_PROG -f -c "falloc $((i * bsize)) $bsize" $testfile \
>> $seqres.full 2>&1
[[ $? != 0 ]] && break
done
Should become:
rm -f $testfile
touch $testfile
echo "* Fallocate unwritten extents"
echo "Fallocate fragmented file"
_scratch_inject_error reduce_max_iextents 1
for i in $(seq 0 2 $((nr_blks - 1))); do
$XFS_IO_PROG -c "falloc $((i * bsize)) $bsize" $testfile \
>> $seqres.full 2>&1
[[ $? != 0 ]] && break
done
_scratch_inject_error reduce_max_iextents 0
With that patched up, xfs/529 passes on 1k block filesystems. I suspect
the other tests in this series (xfs/531, 532, 534, and 535) are going to
need similar patching.
--D
> ---
> tests/xfs/529 | 124 ++++++++++++++++++++++++++++++++++++++++++++++
> tests/xfs/529.out | 11 ++++
> tests/xfs/group | 1 +
> 3 files changed, 136 insertions(+)
> create mode 100755 tests/xfs/529
> create mode 100644 tests/xfs/529.out
>
> diff --git a/tests/xfs/529 b/tests/xfs/529
> new file mode 100755
> index 00000000..dd7019f5
> --- /dev/null
> +++ b/tests/xfs/529
> @@ -0,0 +1,124 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Chandan Babu R. All Rights Reserved.
> +#
> +# FS QA Test 529
> +#
> +# Verify that XFS does not cause bitmap/summary inode fork's extent count to
> +# overflow when growing an the realtime volume of the filesystem.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + _scratch_unmount >> $seqres.full 2>&1
> + test -e "$rtdev" && losetup -d $rtdev >> $seqres.full 2>&1
> + rm -f $tmp.* $TEST_DIR/$seq.rtvol
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/inject
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs xfs
> +# Note that we don't _require_realtime because we synthesize a rt volume
> +# below.
> +_require_test
> +_require_xfs_debug
> +_require_test_program "punch-alternating"
> +_require_xfs_io_error_injection "reduce_max_iextents"
> +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> +_require_scratch_nocheck
> +
> +echo "* Test extending rt inodes"
> +
> +_scratch_mkfs | _filter_mkfs >> $seqres.full 2> $tmp.mkfs
> +. $tmp.mkfs
> +
> +echo "Create fake rt volume"
> +nr_bitmap_blks=25
> +nr_bits=$((nr_bitmap_blks * dbsize * 8))
> +
> +# Realtime extent size has to be atleast 4k in size.
> +if (( $dbsize < 4096 )); then
> + rtextsz=4096
> +else
> + rtextsz=$dbsize
> +fi
> +
> +rtdevsz=$((nr_bits * rtextsz))
> +truncate -s $rtdevsz $TEST_DIR/$seq.rtvol
> +rtdev=$(_create_loop_device $TEST_DIR/$seq.rtvol)
> +
> +echo "Format and mount rt volume"
> +
> +export USE_EXTERNAL=yes
> +export SCRATCH_RTDEV=$rtdev
> +_scratch_mkfs -d size=$((1024 * 1024 * 1024)) -b size=${dbsize} \
> + -r size=${rtextsz},extsize=${rtextsz} >> $seqres.full
> +_try_scratch_mount || _notrun "Couldn't mount fs with synthetic rt volume"
> +
> +echo "Consume free space"
> +fillerdir=$SCRATCH_MNT/fillerdir
> +nr_free_blks=$(stat -f -c '%f' $SCRATCH_MNT)
> +nr_free_blks=$((nr_free_blks * 90 / 100))
> +
> +_fill_fs $((dbsize * nr_free_blks)) $fillerdir $dbsize 0 >> $seqres.full 2>&1
> +
> +echo "Create fragmented filesystem"
> +for dentry in $(ls -1 $fillerdir/); do
> + $here/src/punch-alternating $fillerdir/$dentry >> $seqres.full
> +done
> +
> +echo "Inject reduce_max_iextents error tag"
> +_scratch_inject_error reduce_max_iextents 1
> +
> +echo "Inject bmap_alloc_minlen_extent error tag"
> +_scratch_inject_error bmap_alloc_minlen_extent 1
> +
> +echo "Grow realtime volume"
> +$XFS_GROWFS_PROG -r $SCRATCH_MNT >> $seqres.full 2>&1
> +if [[ $? == 0 ]]; then
> + echo "Growfs succeeded; should have failed."
> + exit 1
> +fi
> +
> +_scratch_unmount >> $seqres.full
> +
> +echo "Verify rbmino's and rsumino's extent count"
> +for rtino in rbmino rsumino; do
> + ino=$(_scratch_xfs_get_metadata_field $rtino "sb 0")
> + echo "$rtino = $ino" >> $seqres.full
> +
> + nextents=$(_scratch_get_iext_count $ino data || \
> + _fail "Unable to obtain inode fork's extent count")
> + if (( $nextents > 10 )); then
> + echo "Extent count overflow check failed: nextents = $nextents"
> + exit 1
> + fi
> +done
> +
> +echo "Check filesystem"
> +_check_xfs_filesystem $SCRATCH_DEV none $rtdev
> +
> +losetup -d $rtdev
> +rm -f $TEST_DIR/$seq.rtvol
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/529.out b/tests/xfs/529.out
> new file mode 100644
> index 00000000..4ee113a4
> --- /dev/null
> +++ b/tests/xfs/529.out
> @@ -0,0 +1,11 @@
> +QA output created by 529
> +* Test extending rt inodes
> +Create fake rt volume
> +Format and mount rt volume
> +Consume free space
> +Create fragmented filesystem
> +Inject reduce_max_iextents error tag
> +Inject bmap_alloc_minlen_extent error tag
> +Grow realtime volume
> +Verify rbmino's and rsumino's extent count
> +Check filesystem
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 2356c4a9..5dff7acb 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -526,3 +526,4 @@
> 526 auto quick mkfs
> 527 auto quick quota
> 528 auto quick quota
> +529 auto quick realtime growfs
> --
> 2.29.2
>
next prev parent reply other threads:[~2021-03-22 17:57 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-09 5:01 [PATCH V6 00/13] xfs: Tests to verify inode fork extent count overflow detection Chandan Babu R
2021-03-09 5:01 ` [PATCH V6 01/13] _check_xfs_filesystem: sync fs before running scrub Chandan Babu R
2021-03-09 5:03 ` Darrick J. Wong
2021-03-10 6:12 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 02/13] common/xfs: Add a helper to get an inode fork's extent count Chandan Babu R
2021-03-09 5:04 ` Darrick J. Wong
2021-03-10 6:12 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 03/13] common/xfs: Add helper to obtain fsxattr field value Chandan Babu R
2021-03-09 5:04 ` Darrick J. Wong
2021-03-10 6:13 ` Allison Henderson
2021-03-10 19:54 ` Darrick J. Wong
2021-03-11 2:54 ` Chandan Babu R
2021-03-11 8:52 ` [PATCH V6.1] " Chandan Babu R
2021-03-11 18:36 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 04/13] xfs: Check for extent overflow when trivally adding a new extent Chandan Babu R
2021-03-10 19:54 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 05/13] xfs: Check for extent overflow when growing realtime bitmap/summary inodes Chandan Babu R
2021-03-10 19:55 ` Allison Henderson
2021-03-22 17:56 ` Darrick J. Wong [this message]
2021-03-23 15:51 ` Chandan Babu R
2021-03-23 20:57 ` Darrick J. Wong
2021-03-24 10:46 ` Chandan Babu R
2021-03-24 14:17 ` Chandan Babu R
2021-03-09 5:01 ` [PATCH V6 06/13] xfs: Check for extent overflow when punching a hole Chandan Babu R
2021-03-10 19:55 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 07/13] xfs: Check for extent overflow when adding/removing xattrs Chandan Babu R
2021-03-10 19:55 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 08/13] xfs: Check for extent overflow when adding/removing dir entries Chandan Babu R
2021-03-10 22:41 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 09/13] xfs: Check for extent overflow when writing to unwritten extent Chandan Babu R
2021-03-10 22:41 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 10/13] xfs: Check for extent overflow when moving extent from cow to data fork Chandan Babu R
2021-03-10 22:41 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 11/13] xfs: Check for extent overflow when remapping an extent Chandan Babu R
2021-03-10 23:49 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 12/13] xfs: Check for extent overflow when swapping extents Chandan Babu R
2021-03-10 23:49 ` Allison Henderson
2021-03-09 5:01 ` [PATCH V6 13/13] xfs: Stress test with bmap_alloc_minlen_extent error tag enabled Chandan Babu R
2021-03-10 23:49 ` Allison Henderson
2021-03-22 18:54 ` Darrick J. Wong
2021-03-23 5:28 ` Chandan Babu R
2021-03-23 11:27 ` Chandan Babu R
2021-03-26 4:05 ` Chandan Babu R
2021-03-26 4:21 ` Darrick J. Wong
2021-04-15 9:33 ` Chandan Babu R
2021-04-17 0:26 ` Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210322175652.GG1670408@magnolia \
--to=djwong@kernel.org \
--cc=chandanrlinux@gmail.com \
--cc=fstests@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).